QPPV.COM KNOWLEDGE ASSESSMENT

CIOMS Working Group XIV: Artificial Intelligence in Pharmacovigilance — Guiding Principles and Practical Implementation

Test your understanding of the concepts covered by CIOMS Working Group XIV: Artificial Intelligence in Pharmacovigilance — Guiding Principles and Practical Implementation.

Question 1 of 10Pass mark: 80%

Question 1 of 10 A large language model produces a plausible case narrative that includes a clinical detail not present in the source. What is the most appropriate control principle?

Accept the detail if it is medically plausible and improves narrative completeness Treat fluent generated text as verified evidence unless a reviewer objects Use generated content as a proposal requiring verification against source information, and do not treat it as evidence in itself Increase the model's confidence threshold so that unsupported details become reliable

Question 2 of 10 Why should evaluation of a generative AI workflow examine the complete human-AI process rather than a single model benchmark?

Because model benchmarks are prohibited for PV systems Because a human reviewer makes model performance irrelevant Because only the user interface determines whether an AI output is accurate Because outputs, human review, workflow controls and changing system components together affect the process's performance

Question 3 of 10 When should acceptance criteria for an AI-supported PV process generally be established?

After routine use, once the organisation has observed enough errors Before routine deployment, so performance and escalation decisions can be assessed against defined expectations Only after an inspection identifies a performance concern Only when the system operates without human review

Question 4 of 10 Which arrangement provides the strongest practical basis for effective human oversight of AI-proposed case fields?

Require reviewers to accept every proposal unless the system displays a low confidence score Record that a human is involved, without changing the review workflow Give trained reviewers access to source information, authority to correct outputs and a way to record their decisions Use a human reviewer only during initial deployment, then rely on the model's historical accuracy

Question 5 of 10 An extraction system meets its overall accuracy target, but review shows that errors often concern negation and temporality. What is the most appropriate next step?

Accept the system because the aggregate target was met Assess the clinical significance of these error categories and strengthen validation or workflow controls as needed Remove negation and temporality from the intended use without documenting the change Replace human review with a higher confidence threshold

Question 6 of 10 Which capability is most relevant to the QPPV organisation's role in governing AI?

Understanding how the AI-supported process affects PV, what can fail, which controls address failures and what evidence shows those controls work Independently developing every model used in the organisation Personally performing all technical validation calculations Approving AI projects solely on the basis of expected processing speed

Question 7 of 10 Which approach to documenting AI in the Pharmacovigilance System Master File is most consistent with the article?

Record every AI experiment, whether or not it forms part of the controlled PV system Exclude all AI-supported processes because the PSMF is limited to manual activities Determine which AI-supported processes materially support the PV system and document them as appropriate under applicable requirements Use the PSMF as a substitute for model validation and change records

Question 8 of 10 How should an organisation approach the EU Artificial Intelligence Act when considering an AI tool for PV?

Assume that every AI tool used by a pharmaceutical company has the same classification Treat CIOMS principles as a substitute for assessing the Act Assess the Act alongside medicines, data-protection and quality obligations, based on the system's characteristics and intended use Apply the Act only if the AI system makes a final medical decision without human involvement

Question 9 of 10 Why might a high overall accuracy result be insufficient to establish that an AI extraction tool is fit for PV use?

Overall accuracy is never a useful measure for any AI application PV systems must use the same performance threshold regardless of task High accuracy demonstrates that all clinically important errors will be detected Aggregate performance can conceal errors in important fields or error types, including those with serious safety implications

Question 10 of 10 Which statement best distinguishes validation from routine performance monitoring?

Validation measures user satisfaction, while monitoring confirms legal classification Validation establishes evidence for a defined use before deployment; monitoring checks whether performance remains acceptable after deployment Validation is needed only for vendor-developed models, while monitoring is needed only for in-house models Validation and monitoring are interchangeable terms for testing a model once

QPPV.com

CIOMS Working Group XIV: Artificial Intelligence in Pharmacovigilance — Guiding Principles and Practical Implementation

Cookie Settings