CIOMS Working Group XIV: Artificial Intelligence in Pharmacovigilance — Guiding Principles and Practical Implementation

Artificial intelligence is increasingly being considered for pharmacovigilance activities that range from information extraction and case processing to duplicate detection, translation, signal detection and information synthesis. The attraction is understandable: safety organisations work with large volumes of heterogeneous information, much of which must be transformed before expert assessment can begin. AI can potentially reduce repetitive work, identify patterns, and make large information sets more accessible to pharmacovigilance professionals. At the same time, an incorrect AI output can enter a regulated safety process and influence a decision about an individual patient, a safety signal, a regulatory submission or a benefit-risk assessment.

CIOMS Working Group XIV addressed this problem directly. Its report, Artificial Intelligence in Pharmacovigilance, was published in December 2025 after a multi-year working-group process involving experts from regulatory authorities, industry, academia and international organisations. Rather than prescribing a particular technology, model architecture or validation algorithm, the report establishes seven principles intended to remain useful as the technology changes: a risk-based approach, human oversight, validity and robustness, transparency, data privacy, fairness and equity, and governance and accountability. The principles are intended to apply across the lifecycle of AI used in pharmacovigilance rather than only at the point of deployment. [1–3]

Purpose and Scope

The central purpose of CIOMS Working Group XIV is not to declare that AI is either suitable or unsuitable for pharmacovigilance. It provides a framework for deciding how AI should be developed, evaluated and used when it is introduced into a safety process.

This distinction matters because “AI in pharmacovigilance” is not one use case. A system that proposes a MedDRA term for a human reviewer, a model that identifies potentially duplicate cases, a large language model that extracts information from a source document, and a model that contributes to signal detection have different failure modes and different consequences when those failures occur. Treating them as one class of technology encourages either excessive controls for low-risk applications or insufficient controls for high-risk ones.

CIOMS therefore takes a risk-based approach. The level of oversight, performance evidence, transparency and governance should reflect the AI system, the task for which it is used, the degree of autonomy, the consequences of error and the ability of humans to detect and correct an incorrect output. The same model can consequently require different controls when used for different PV tasks. [1,2]

The report is also deliberately broader than generative AI. Its scope encompasses AI and intelligence-augmentation approaches used in pharmacovigilance, including conventional machine-learning approaches and systems that support or automate parts of ICSR processing and signal management. Generative AI and large language models receive particular attention because they introduce additional concerns such as variable outputs, hallucination, prompt and data leakage, model updates and uncertain persistence or secondary use of submitted information. [2]

The practical implication is that an organisation should begin with the PV process and the intended decision, not with the technology. “We want to use an LLM” is not a sufficient use-case definition. A defensible definition identifies the task, the input data, the output, the intended user, the degree of automation, the decisions the output may influence and the controls that prevent an erroneous output from silently becoming a safety conclusion.

Why AI Requires a Pharmacovigilance-Specific Framework

Pharmacovigilance is unusually dependent on imperfect, heterogeneous and context-rich information. An ICSR may contain clinical observations, laboratory findings, chronology, treatment history, concomitant medicines, reporter interpretation and uncertainty. A safety signal may depend on relationships that are not explicit in individual records. The value of an automated output therefore cannot be judged solely by generic technical accuracy.

For example, an extraction system that identifies the correct adverse-event term in 98% of test fields may still be unsuitable if the remaining errors systematically concern serious reactions. Similarly, a signal-detection model that performs well on common events may have inadequate sensitivity for rare but clinically important patterns. A translation system can produce linguistically fluent text while altering a clinically relevant qualification. These are not simply software defects; they can alter the evidence on which pharmacovigilance decisions are made.

CIOMS consequently treats AI as part of a socio-technical system. Model performance, source data, workflow design, human review, user behaviour, documentation and governance interact. Human involvement is not a generic safety slogan: its usefulness depends on whether reviewers can understand the output, detect errors, challenge it and take an appropriate alternative action.

The framework also recognises the risk of automation bias. A reviewer may give excessive weight to an apparently authoritative machine output, particularly when the system is fast, consistent or presented with a confidence score. Human oversight therefore needs to be designed rather than merely declared. A nominal “human review” step is weak control if the reviewer cannot reconstruct the basis of the AI output or is not trained to recognise its characteristic failure modes. [2]

Regulatory Status and Relationship to EU Pharmacovigilance Requirements

The CIOMS Working Group XIV report is international consensus guidance. It is not EU legislation, an EMA legally binding regulation, or an EU GVP module. Its principles should therefore not be described as standalone statutory requirements.

The legal and regulatory obligations governing a marketing authorisation holder's pharmacovigilance system remain applicable regardless of whether a particular activity is performed manually, through conventional automation, or with AI. In the EU, AI does not transfer responsibility for the pharmacovigilance system from the marketing authorisation holder to a software provider. Where an AI-supported process contributes to a regulated PV activity, the organisation remains responsible for ensuring that the overall process is appropriate, controlled and capable of meeting applicable requirements.

The European medicines regulatory network is itself developing its approach to AI. EMA's 2024 reflection paper addresses the use of AI and machine learning across the medicinal-product lifecycle, while the HMA-EMA Network Data Steering Group workplan for 2026–2028 includes development of AI guidance for clinical development and pharmacovigilance. PRAC's 2026 work plan likewise includes preparatory work on AI guidance in pharmacovigilance. These initiatives demonstrate that the regulatory environment is evolving; they should not be interpreted as making the CIOMS principles legally binding. [4–7]

The EU Artificial Intelligence Act adds another layer of law that may apply depending on the characteristics and intended use of an AI system. Its classification and obligations are determined by the Regulation and the particular use case, not simply by the fact that an AI tool is used inside a pharmaceutical company. The AI Act should therefore be assessed alongside medicines law, data-protection law and existing quality-system obligations rather than treated as a replacement for them. [8]

For a QPPV organisation, the safest conceptual model is therefore:

This layered approach prevents two opposite errors: treating CIOMS as if it were legislation, or assuming that because CIOMS is not legislation its principles have no operational relevance.

From AI Project to Controlled Pharmacovigilance Process

An AI initiative becomes a pharmacovigilance governance issue when its output can affect a PV activity. The first control is therefore to describe the intended use precisely.

A useful specification identifies at least the process, task, input, output, user, decision context, autonomy and failure consequence. For example, “AI for case processing” is too broad. “An AI model proposes structured extraction of event, medicinal product, dose and chronology fields from incoming source documents; a trained case processor reviews every proposed field before database entry” is sufficiently specific to support a meaningful risk assessment.

The specification should also distinguish decision support from decision automation. A system that presents a ranked list for expert review creates a different control environment from a system that automatically determines whether an ICSR is valid, closes a potential duplicate, or suppresses an item from downstream processing.

This distinction becomes the foundation for the seven CIOMS principles. Risk determines the required oversight; the intended use determines what must be validated; the validation strategy determines what evidence is necessary; transparency makes the system and its limitations intelligible; privacy controls protect the underlying safety data; fairness controls address uneven performance; and governance assigns responsibility for the complete lifecycle.

Figure 1. The CIOMS Working Group XIV framework is best understood as an interconnected control system rather than seven independent checks. Risk assessment determines the intensity of controls across development, deployment and routine use; governance maintains those controls throughout the lifecycle.

The Seven Guiding Principles

The seven principles are complementary. They should not be implemented as seven isolated workstreams because a weakness in one principle can change the requirements under another. A high-risk use case, for example, generally demands stronger human oversight and more extensive performance evidence. A poorly represented data set creates both a validity problem and a fairness problem. Limited model transparency may increase the level of human review needed because users have less ability to interrogate the output.

1. Risk-Based Approach

The risk-based principle is the organising concept for the framework. CIOMS recommends that organisations identify and prioritise risks associated with the AI system and manage them proportionately throughout its lifecycle.

The assessment should consider more than the model's technical characteristics. It should examine the context in which the model is used, the likelihood and detectability of failure, and the potential impact. A useful assessment therefore asks what happens if the AI is wrong, how quickly the error could be detected, whether a human can correct it, and whether the error could propagate into another PV process.

Consider two systems. The first suggests possible duplicate cases to a trained reviewer, who confirms or rejects every suggestion. The second automatically suppresses records classified as duplicates. Both may use similar machine-learning methods, but the consequences of an incorrect output differ substantially. The second use case requires much stronger evidence and controls because an undetected false duplicate could remove safety information from subsequent assessment.

Risk assessment should also remain live. A model may be acceptable for a limited pilot but become higher risk when its scope expands, the population changes, the data source changes, the level of automation increases, or a new model version is introduced. Risk is therefore a lifecycle attribute, not a one-time project document. [1,2]

2. Human Oversight

CIOMS distinguishes several forms of human involvement. In a human-in-the-loop arrangement, the human participates in producing the final decision or output. In a human-on-the-loop arrangement, the system produces an output autonomously and a human checks it. Human-in-command represents a stronger governance concept in which humans retain authority over the system and its use.

These labels are useful only when translated into actual controls. “Human in the loop” does not by itself establish effective oversight. The reviewer must have sufficient competence, time, information and authority to challenge the system. The workflow must make it possible to correct an output rather than simply accept it. Reviewers should also understand known failure modes, including automation bias and the tendency to over-trust fluent or apparently confident outputs.

For PV, the oversight design should answer five operational questions:

1. Which AI outputs require mandatory human review?
2. What evidence is presented to the reviewer?
3. What decisions can the reviewer override?
4. How are overrides and corrections recorded?
5. What happens when performance deteriorates or the system becomes unavailable?

The answer will differ by use case. A language model used to draft a case narrative may require line-by-line verification against source information. A duplicate-detection system may require review of the candidate pair and the features supporting the match. A signal-detection model may require medical review of the underlying cases and alternative explanations rather than simply review of the model score.

Human oversight also changes workforce requirements. PV professionals may need sufficient AI literacy to understand model limitations, performance measures, data provenance and appropriate use. Conversely, technical teams need sufficient understanding of PV concepts to recognise why an apparently minor extraction error can be clinically important. CIOMS therefore treats training and organisational readiness as part of responsible implementation rather than as an afterthought. [2]

3. Validity and Robustness

Validity asks whether the system performs adequately for its intended purpose. Robustness asks whether that performance remains acceptable under relevant variation.

The starting point is the intended use. There is no universal threshold such as “95% accuracy” that makes an AI system fit for pharmacovigilance. The appropriate performance measures depend on the task and the consequences of different error types.

For an extraction task, sensitivity and precision for clinically important fields may matter more than aggregate accuracy. For duplicate detection, false positives and false negatives have different operational consequences. For signal-support tools, the usefulness of ranked outputs may depend on sensitivity for uncommon patterns, reviewer workload and the quality of the underlying evidence. For generative systems, factual accuracy, completeness, unsupported statements and reproducibility of outputs may need to be assessed separately.

Evaluation should use data representative of the intended deployment environment. CIOMS emphasises qualitative review as a complement to quantitative metrics and draws attention to subgroup analyses, sensitivity analyses and the challenge of evaluating low-prevalence events. A test set dominated by common cases can conceal poor performance on precisely the cases that matter most to safety surveillance.

External validation is particularly important when a system is moved between populations, products, therapeutic areas, languages, databases or organisations. A model trained on one data-generating environment may perform differently after deployment because terminology, reporting behaviour, source-document structure or case complexity has changed.

Generative AI adds further challenges. Outputs may vary, models may be updated by providers, and fluent language can obscure factual errors. Evaluation should therefore examine the complete human-AI workflow rather than treating a single model benchmark as proof of fitness for production. CIOMS also emphasises reproducibility and ongoing performance evaluation where models or their operating environments change. [1,2]

4. Transparency

Transparency makes an AI system inspectable enough for stakeholders to understand what it is intended to do, how it is being used and what its limitations are.

For a PV organisation, useful transparency includes the intended use, model or system characteristics, input and output definitions, training and validation data where appropriate, human interaction, performance evidence, known limitations and version information. Transparency does not necessarily mean disclosing proprietary source code. It means providing enough information to support appropriate use, evaluation, challenge and governance.

Explainability is related but not synonymous with transparency. A model may have an explanation method without being fit for purpose. Conversely, a transparent description of intended use and performance may be possible even when the internal model is difficult to interpret. Post-hoc explainability techniques can themselves have limitations and should not be treated as proof that the model actually reasoned through the pathway suggested by the explanation.

For regulated PV processes, transparency should extend to operational provenance. A reviewer or auditor should be able to determine which model or service version produced an output, what input was provided, what relevant configuration was used, whether a human changed the output and what final decision entered the controlled process.

Vendor-provided AI creates an additional transparency question. A contract may allocate technical responsibilities to a supplier, but the PV organisation still needs sufficient information to determine whether the service is fit for its intended use and appropriately controlled. Where a vendor cannot provide meaningful information about performance, changes, limitations or data handling, the organisation's ability to demonstrate effective oversight is correspondingly weakened. [2]

5. Data Privacy

Pharmacovigilance information can contain sensitive health data and, depending on the source and jurisdiction, other personal information. AI can increase privacy risk because it may enable large-scale linkage, broader access, automated inference or secondary processing of information.

CIOMS therefore emphasises that existing privacy controls may need to be reassessed when AI is introduced. This is especially relevant to generative AI services, where prompts and uploaded documents may be processed outside the organisation's established safety environment or may be subject to retention or secondary-use arrangements that the PV team has not considered.

Data minimisation, appropriate de-identification or anonymisation where feasible, access control, encryption, controlled environments and documented data flows are practical safeguards. A data-protection impact assessment may be appropriate where required by applicable law and where the processing creates relevant risk.

Privacy must be assessed across the complete data lifecycle. The question is not only whether a model can access an ICSR. It includes where the data are copied, whether prompts and outputs are retained, who can access logs, whether data are used for model training, how data are transferred to vendors, and what happens when the service is discontinued.

For an EU organisation, these controls operate alongside the GDPR and other applicable data-protection requirements. CIOMS does not replace those legal requirements; it highlights why AI can expose weaknesses in existing PV data-handling arrangements. [2,8]

6. Fairness and Equity

Fairness and equity are particularly important in pharmacovigilance because the underlying safety data are not necessarily representative of all populations exposed to a medicine. Reporting rates vary between countries and healthcare systems. Some populations may be underrepresented in clinical studies or spontaneous-reporting data. Paediatric, older, pregnant, geographically distinct or otherwise less represented groups can consequently provide less information for model development and evaluation.

An AI system can reproduce or amplify these limitations. A model trained on historically imbalanced data may perform well for the majority population while underperforming for a subgroup. If the organisation evaluates only overall performance, the disparity may remain hidden.

Fairness assessment therefore begins with data representation. The organisation should understand which populations are present in the development and test data, which are missing or sparsely represented, and whether performance differs materially between relevant subgroups. Where feasible, subgroup analysis and sensitivity analysis should be incorporated into validation.

The objective is not to require identical model performance in every conceivable population. The objective is to identify clinically and operationally meaningful disparities, understand their causes and incorporate them into the risk assessment and intended-use decision. If evidence is insufficient for a population, the limitation should be documented rather than concealed by an overall performance figure.

Fairness also interacts with human oversight. A model known to perform less reliably for a subgroup may require additional review or may be inappropriate for that use altogether. Thus fairness is not simply an ethical appendix to technical validation; it can change the conditions under which the system is safe to use. [2]

7. Governance and Accountability

Governance connects the other principles to organisational responsibility. CIOMS describes governance across the AI lifecycle, from requirements specification and development through pre-deployment, post-deployment and routine use. The organisation should be able to identify who is accountable at each stage and how the system can be corrected, suspended or disabled when necessary.

A useful PV governance model separates at least four questions. First, who owns the business process? Second, who is accountable for the AI system? Third, who validates and monitors its performance? Fourth, who has authority to stop its use when a material problem is identified?

These roles may be distributed among PV, quality, IT, data science, privacy, procurement and a vendor. Distribution is not itself a problem; ambiguity is. A vendor may develop or operate a model, but outsourcing a technical activity does not automatically outsource the organisation's regulatory accountability for the PV process.

Traceability and version control are especially important. A change in model, prompt configuration, training data, retrieval corpus, vendor service or supporting software can change output behaviour. The organisation therefore needs a controlled method for identifying versions, assessing changes, determining whether revalidation is required and retaining evidence of the decision.

Business continuity also belongs within AI governance. If a critical PV process depends on an AI service, the organisation needs a controlled fallback when the service becomes unavailable or its performance is suspended. The fallback does not necessarily have to reproduce the automated process; it must preserve the ability of the PV system to continue meeting applicable obligations.

Translating the Principles into an AI Lifecycle

The seven principles become operational when mapped to a lifecycle. The following sequence is a practical implementation model derived from CIOMS rather than a mandatory CIOMS procedure.

This lifecycle view also clarifies an important distinction between validation and monitoring. Validation establishes evidence before a defined use. Monitoring asks whether that evidence remains representative after deployment. A model can pass validation and later degrade because the source population changes, a vendor changes the underlying model, reporting patterns evolve, or the organisation begins using the system outside its original scope.

For this reason, acceptance criteria should be defined before routine deployment. They should include not only aggregate performance but also clinically meaningful error categories, subgroup performance where relevant, workload and human-review effectiveness, and criteria for escalation or suspension.

Applying CIOMS XIV to Common PV Use Cases

The CIOMS report includes use cases illustrating how its principles can be applied. These examples are valuable because they demonstrate that the control framework changes with the task. They should not be read as blanket endorsements of a particular technology.

AI-Assisted Case Processing

An AI system may extract patient characteristics, medicinal products, adverse events, dates, doses or other information from source documents. The main risk is not merely extraction error; it is the possibility that an incorrect extraction becomes an apparently authoritative structured field and is not detected.

A controlled implementation should preserve access to the source information, make the AI proposal distinguishable from verified data, allow the processor to correct it, and retain an appropriate audit trail. Validation should examine clinically important fields separately rather than relying only on overall extraction accuracy. Particular attention is warranted for negation, temporality, uncertainty, dose relationships and distinctions between the reporter's statement and a medical interpretation.

Generative systems require additional safeguards because a fluent answer can contain information that was not present in the source. A PV workflow should therefore treat generated content as an output requiring verification, not as evidence in its own right.

Duplicate Detection

Duplicate detection illustrates why false positives and false negatives must be assessed separately. A false negative leaves duplicate information in the database; a false positive may cause distinct patient reports to be incorrectly combined or one report to be suppressed.

Human review can be particularly valuable when the system presents the candidate records and the information supporting the predicted relationship. The organisation should monitor not only model performance but also the behaviour of reviewers, because repeated exposure to high-quality suggestions can create automation bias.

Translation

AI-assisted translation may help make multilingual safety information available more quickly. However, clinically important meaning can be altered by translation errors, especially around negation, timing, severity, anatomical terminology and uncertainty.

Validation should therefore assess clinically relevant content rather than general linguistic fluency alone. Where human review is required, the reviewer should have sufficient language and PV competence to identify safety-relevant mistranslation.

Signal Detection and Signal Management Support

AI may support signal detection by prioritising information, identifying patterns, reducing repetitive review or combining data sources. The output remains an analytical input rather than a regulatory conclusion.

The key control is preservation of expert assessment. Signal evaluation requires consideration of alternative explanations, data quality, biological plausibility, exposure, clinical context and the totality of evidence. An algorithmic score can contribute to this process but does not remove the need for medical and scientific interpretation.

Information Synthesis and Large Language Models

Large language models can retrieve and synthesise information from safety-document collections. The principal risks include hallucination, omission, inappropriate inference, outdated source material and leakage of sensitive information.

A production system should constrain the information sources where appropriate, preserve links or references to the underlying documents, distinguish extracted facts from generated synthesis, and require human verification for outputs used in regulated decisions. The model's ability to produce a plausible narrative should never be confused with evidence that the narrative is factually complete.

Quality Management and Evidence

Introducing AI into pharmacovigilance does not create a separate quality system. Instead, AI-related controls should connect to the organisation's existing quality-management framework.

The evidence should be sufficient to reconstruct how the AI-supported process operated at the relevant point in time. Depending on the use case, this can include the approved intended use, risk assessment, model or service version, data specifications, validation evidence, acceptance criteria, user training, monitoring results, deviations, changes, overrides, vendor records and decisions to continue, restrict or suspend use.

The exact documentation set should be proportionate to the risk and applicable requirements. CIOMS provides principles rather than a universal document template. A low-risk analytical tool used for exploratory work may not require the same controls as an AI system embedded directly in ICSR processing.

The Pharmacovigilance System Master File is also relevant where AI components materially support the pharmacovigilance system and fall within the information that should be documented there under applicable requirements. CIOMS specifically highlights PSMF documentation as an example of how AI governance connects to established PV system documentation. GVP Module II requires the PSMF to describe relevant computerised systems and databases used for pharmacovigilance, including their functionality, operational responsibility and fitness-for-purpose assessment, with validation status and related controls described as applicable. The correct approach is therefore not to place every AI experiment into the PSMF, but to determine which AI-supported systems form part of the controlled PV system and document them appropriately. [2,9]

Vendor and Outsourced AI Services

Vendor involvement changes the control boundary but not the need for accountability. A vendor may provide a model, application, cloud infrastructure, translation service or managed AI platform. The PV organisation should understand which elements it controls and which depend on the supplier.

Due diligence should address the intended use, data flows, security, privacy, performance evidence, model and service changes, incident handling, availability, audit rights where appropriate, retention and deletion, and the supplier's ability to provide information needed for PV oversight.

The most important contractual question is not simply whether the vendor claims that its AI is “validated.” Validation is context-dependent. A model may have been evaluated for a generic task while the MAH uses it in a specific PV workflow with different data, populations, languages and consequences. The organisation must therefore establish fitness for its own intended use.

A vendor update should be treated as a potential change to the controlled system. Depending on the nature of the change, the organisation may need impact assessment, regression testing, targeted revalidation, updated risk assessment or renewed approval before the new version is used in production.

Change Management and Model Drift

Traditional software change control is necessary but may not be sufficient for AI. Some systems change because a new model version is deliberately deployed; others can change through retraining, updated reference data, vendor-side model changes or alterations to prompts and retrieval sources.

The organisation should define what constitutes a controlled change for the particular system. A prompt template can be part of the functional configuration of a generative-AI workflow. A retrieval corpus can alter the information available to a model. A vendor's silent model upgrade can change output behaviour without any local code change.

Monitoring should therefore look for evidence of performance drift as well as technical availability. Useful indicators depend on the task but may include error rates, clinically important error categories, override rates, subgroup performance, unexpected output patterns, reviewer complaints and deviations from the intended workflow.

The response to deterioration should be predetermined. Possible actions include increased human review, restriction of the use case, rollback to a previous version, temporary suspension, additional validation or redefinition of the intended use.

Inspection Perspective

An inspector evaluating an AI-supported PV process would reasonably be interested in whether the organisation can demonstrate control over the complete process, not whether it has purchased an AI product.

Illustrative inspection questions include:

These are illustrative questions, not claims about specific published inspection findings. Their purpose is to show how the CIOMS principles translate into evidence that an inspector could evaluate.

A weak system is often recognisable by a mismatch between the sophistication of the technology and the maturity of the controls around it. An organisation may have a technically impressive model but no defined intended use, no acceptance criteria, no controlled version history, no monitoring plan and no clear owner. Conversely, a relatively simple model can be responsibly used when its limitations are known and the workflow controls those limitations.

QPPV and Safety-System Governance

The QPPV does not need to become a data scientist to govern AI effectively. The critical responsibility is understanding how the AI-supported process affects the pharmacovigilance system and ensuring that appropriate expertise and controls exist.

This requires the QPPV organisation to be able to answer four questions. What safety activity is affected? What could go wrong? What controls prevent or detect those failures? What evidence demonstrates that the controls continue to work?

The answers should connect AI governance to established PV governance rather than create a parallel universe. For example, if an AI system supports ICSR processing, its risk assessment should connect to case-processing quality controls. If it supports signal management, its performance should be considered within the signal-management governance framework. If it changes the way safety information is obtained or interpreted, the relevant procedures, training, reconciliation and oversight arrangements should be examined.

The QPPV should also be able to challenge an AI initiative that is justified mainly by efficiency. Faster processing is not itself a pharmacovigilance outcome. The relevant question is whether the process remains accurate, timely, traceable and capable of supporting sound safety decisions. CIOMS's emphasis on risk, human oversight and governance provides a useful basis for that challenge.

Common Implementation Failures

Several failure modes follow naturally when the seven principles are not treated as an integrated system.

Starting with the technology rather than the PV task. An organisation selects an LLM or machine-learning product before defining the problem, resulting in unclear intended use and weak acceptance criteria.

Using generic accuracy as proof of fitness. Overall performance can conceal clinically important errors, subgroup underperformance or failure on rare events.

Treating human review as a checkbox. A reviewer who cannot see the evidence behind an output, lacks time to challenge it or is trained only to accept recommendations does not provide effective oversight.

Ignoring model and vendor change. A system can change even when the organisation's own application code does not. New model versions, retrieval sources, prompts or vendor configurations may alter behaviour.

Allowing sensitive data into uncontrolled AI services. A public or externally hosted service may create data-retention, transfer or secondary-use risks that were not present in the original PV workflow.

Failing to preserve provenance. If the organisation cannot identify the model version, input, output, reviewer action and final decision, it may be unable to reconstruct how a safety decision was supported.

Assuming the vendor owns the risk. A supplier can provide technical controls and evidence, but the MAH must still govern the PV process for which the technology is used.

Expanding the use case without reassessment. A tool validated for document extraction may later be used for medical assessment or signal prioritisation without a corresponding change assessment. This changes the risk and intended use and should trigger reassessment.

Practical Implementation Checklist

Before placing an AI-supported PV process into routine use, an organisation should be able to demonstrate that it has:

• defined the intended PV use and the decision context;
• assessed the consequences of incorrect or incomplete output;
• established the required human-oversight model;
• defined task-appropriate performance measures and acceptance criteria;
• tested representative data and relevant subgroups where appropriate;
• reviewed qualitative examples of correct and incorrect output;
• documented known limitations and intended-use boundaries;
• established privacy and information-security controls;
• defined fairness and equity considerations relevant to the data and population;
• assigned accountable owners across the AI lifecycle;
• established model, configuration and vendor version control;
• defined routine performance monitoring and escalation thresholds;
• established change-control and revalidation criteria;
• documented fallback arrangements for service failure or suspension;
• connected the AI process to the applicable PV quality system and records;
• determined what AI-related information belongs in controlled PV documentation, including the PSMF where applicable.

The checklist is an operational interpretation of the CIOMS principles, not a CIOMS-mandated checklist. The appropriate controls depend on the use case, applicable law and the organisation's quality system.

Key Takeaways

CIOMS Working Group XIV provides a durable way to think about AI in pharmacovigilance because it does not depend on a particular model technology. The framework begins with risk and follows that risk through human oversight, evidence of validity and robustness, transparency, privacy, fairness and governance.

The most important practical consequence is that AI should be governed as part of the pharmacovigilance process in which it operates. A model is not independently “safe for PV.” It is fit or unfit for a defined use, under defined conditions, with defined controls and evidence.

The seven principles are also interdependent. Higher-risk uses generally require stronger oversight and validation. Poor data representation can create both performance and fairness problems. Limited transparency can increase the need for human scrutiny. Vendor or model changes can require renewed assessment. Governance provides the mechanism through which these relationships remain controlled over time.

For QPPV organisations, the resulting standard is straightforward: understand the PV task, define what the AI is allowed to do, demonstrate that it performs adequately for that task, retain meaningful human authority, protect the underlying data, monitor for deterioration and maintain clear accountability throughout the lifecycle.

References

1. Council for International Organizations of Medical Sciences (CIOMS). Artificial Intelligence in Pharmacovigilance — Report of the CIOMS Working Group XIV. Geneva: CIOMS; 2025. DOI: 10.56759/cdob6397. Available from: https://cioms.ch/publications/
2. Norén GN, Durand J, Kara V, et al.; CIOMS Working Group XIV. Artificial Intelligence in Pharmacovigilance: Guiding Principles from the CIOMS Working Group XIV. Drug Safety. 2026. doi:10.1007/s40264-026-01708-z. Available from: https://doi.org/10.1007/s40264-026-01708-z
3. CIOMS. Working Group XIV — Artificial Intelligence in Pharmacovigilance. Available from: https://cioms.ch/working_groups/working-group-xiv-artificial-intelligence-in-pharmacovigilance/
4. European Medicines Agency. Reflection paper on the use of artificial intelligence (AI) in the medicinal product lifecycle. EMA/CHMP/CVMP/83833/2023. Adopted September 2024. Available from: https://www.ema.europa.eu/en/scientific-guidelines/use-artificial-intelligence-ai-medicinal-product-lifecycle
5. European Medicines Agency and Heads of Medicines Agencies. Data and artificial intelligence in medicines regulation to 2028. Network Data Steering Group workplan, 2026–2028. Available from: https://www.ema.europa.eu/en/about-us/how-we-work/data-regulation-big-data-other-sources/network-data-steering-group-ndsg
6. European Medicines Agency. PRAC work plan 2026. Available from: https://www.ema.europa.eu/en/documents/work-programme/prac-work-plan-2026_en.pdf
7. European Medicines Agency. Artificial intelligence. Current EMA overview of AI work in medicines regulation, including the Network Data Steering Group workplan and AI guidance initiatives. Available from: https://www.ema.europa.eu/en/about-us/how-we-work/data-regulation-big-data-other-sources/artificial-intelligence
8. European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), as amended. EUR-Lex. Available from: https://eur-lex.europa.eu/eli/reg/2024/1689
9. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module II — Pharmacovigilance system master file (Rev. 2). EMA/816573/2011. Available from: https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-ii-pharmacovigilance-system-master-file-rev-2_en.pdf

Regulatory Note

The CIOMS Working Group XIV report is consensus guidance and should not be presented as binding EU pharmacovigilance law. Applicable medicines legislation, GVP requirements, data-protection law, the EU Artificial Intelligence Act where applicable, contractual obligations and the organisation's quality system determine the mandatory control environment for a specific AI implementation. CIOMS principles provide a useful framework for responsible design, evaluation and governance within that environment. Requirements and guidance may evolve as EMA, HMA, national competent authorities and other regulators develop more specific approaches to AI in pharmacovigilance.