QPPV.COM KNOWLEDGE ASSESSMENT

MHRA Beyond ADMET: AI for Medicines Safety and the Regulatory Sandbox — Knowledge Assessment

Test your understanding of the concepts covered by MHRA Beyond ADMET: AI for Medicines Safety and the Regulatory Sandbox.

Question 1 of 10Pass mark: 80%

Question 1 of 10 A model ranks patients with higher observed risk above those with lower risk, but systematically overestimates the absolute probability of an adverse outcome. Which interpretation is most appropriate?

The model's ranking ability may be useful, but poor calibration can make its probability estimates misleading. Good ranking makes calibration irrelevant for regulatory or clinical decisions. Overestimation proves that the model has no discriminatory ability. The probability estimates can be used without qualification if the dataset was large.

Question 2 of 10 A team proposes replacing an established safety study with an AI prediction because the model has promising published performance. What is the most appropriate regulatory interpretation?

Published performance automatically permits replacement when the model predicts the same endpoint. Replacement is established by the sandbox if the model is discussed in a workshop. The model's performance and relevance must be evaluated for the proposed context; the sandbox does not itself remove existing evidence requirements. Any AI prediction may replace a study if it is accompanied by a mechanistic explanation.

Question 3 of 10 A team claims that its model is suitable for use in a new healthcare system because the new system's patients are represented in the original training data. What further evidence may be particularly useful?

External or temporal validation in the relevant setting, because representation alone does not establish generalisability. A larger training set from the original system, which makes validation in the new setting unnecessary. A comparison of model architecture names across systems. Confirmation that the new system uses the same drug names.

Question 4 of 10 A developer argues that a very large real-world dataset is sufficient to establish model validity. Which response best reflects the article's approach?

Large datasets remove the need to assess confounding if the model uses many variables. Dataset size is useful, but provenance, completeness, measurement quality, representativeness and fitness for the specific purpose also matter. Real-world data should not be used for safety models because they are observational. A large dataset makes outcome validation unnecessary.

Question 5 of 10 A model predicts few cases of a rare but severe toxicity. The developer interprets this as evidence that the toxicity is unlikely. What is the most important qualification?

A negative prediction is reassuring only if the model has adequate sensitivity and relevant validation for that rare outcome. Rare outcomes do not require validation because there are few cases to assess. The prediction is conclusive if specificity is high. A low number of predictions proves that the training data were representative.

Question 6 of 10 A model has high apparent accuracy on a test set, but some patients have repeated records in both the training and test sets. What is the principal concern?

The model may be poorly calibrated because repeated records always distort predicted probabilities. The test results may be optimistic because information from the evaluation population has leaked into model development. The model cannot be externally validated if it uses patient-level records. The model's sensitivity must be higher than its specificity.

Question 7 of 10 A model identifies a possible toxicity association using linked clinical and real-world data. How should pharmacovigilance professionals treat that output?

As a causal conclusion, because integration across data sources resolves alternative explanations. As a hypothesis to assess alongside chronology, biological plausibility, other safety evidence and alternative explanations. As a confirmed signal if the model's output is reproducible. As a substitute for review of spontaneous reports and epidemiological evidence.

Question 8 of 10 A company tells its governance committee that participation in the MHRA Beyond ADMET sandbox means its AI safety model is qualified for use in a marketing-authorisation application. What is the most appropriate response?

Participation establishes regulatory acceptance for the model's stated endpoint, provided the model was demonstrated to the MHRA. The sandbox supports exploration of evidence and regulatory acceptability; participation does not itself qualify the model or establish general acceptance. Participation allows the company to replace conventional safety evidence with model predictions for the population studied. The model is qualified if its performance exceeds that of the company's existing preclinical assays.

Question 9 of 10 An AI model reports strong performance for a composite endpoint labelled “liver toxicity.” What should a regulatory reviewer clarify first when assessing the result?

Whether the model architecture is the most recently published architecture. Whether the composite includes the largest possible number of liver-related records. What clinically or scientifically defined outcome the model predicts and whether that outcome fits the proposed decision. Whether the model can provide a visual explanation for each prediction.

Question 10 of 10 A model combines chemical, omics, clinical-trial and healthcare-record data. Why does combining these sources make data provenance especially important?

Integrated models cannot be validated unless all data come from one source. Reviewers need to understand the sources, transformations and drivers of performance, including whether those inputs will exist in the intended setting. Combining data automatically removes source-specific bias. Provenance matters only for personally identifiable information, not for scientific interpretation.

MHRA Beyond ADMET: AI for Medicines Safety and the Regulatory Sandbox — Knowledge Assessment

Cookie Settings