EudraVigilance Metrics and KPIs for QPPVs

A QPPV does not need a dashboard containing every available operational statistic. The useful question is whether the metrics reveal, early enough to act, whether pharmacovigilance processes that depend on EudraVigilance remain compliant and effective.

GVP Module I explicitly recognises performance indicators as a means of continuously monitoring pharmacovigilance activities. It does not prescribe a universal set of EudraVigilance KPIs or fixed internal alert thresholds. That distinction is central to good governance. Regulatory timelines such as the 15-day and 90-day ICSR submission requirements are binding within their scope; a company target such as "rejections resolved within two business days" is an internal control unless a specific regulatory requirement says otherwise.

Purpose of EudraVigilance Metrics

Metrics should support three levels of oversight:

1. compliance — are applicable obligations being met?
2. control — are the processes detecting and correcting exceptions before they become systemic?
3. effectiveness — is the system producing reliable safety information for regulatory reporting and signal management?

A dashboard that shows workload but cannot answer those questions is not an effective oversight tool.

Regulatory Basis for Performance Monitoring

GVP Module I states that organisations may use performance indicators to continuously monitor the good performance of pharmacovigilance activities. It also requires management review of the pharmacovigilance quality system and identifies compliance monitoring as part of evaluating system performance and effectiveness.

For EudraVigilance-related activities, the underlying regulatory requirements arise from the legislation and GVP modules governing ICSR reporting, signal management and the pharmacovigilance quality system. Metrics translate those requirements into observable performance; they do not create new legal obligations.

The distinction can be expressed simply:

The last two may be very useful, but they should not be presented as EMA-mandated thresholds.

Designing a Useful KPI

A strong KPI has a defined numerator, denominator, data source, owner, frequency, threshold or interpretation rule, and escalation path. It should also be resistant to gaming.

For example, an on-time reporting rate is weak if "receipt date" means safety-database entry while cases may have been held in an affiliate mailbox earlier. The KPI definition must therefore use the actual regulatory clock start.

Leading and lagging indicators

Lagging indicators show that a failure has already occurred: late submissions, rejected cases, overdue signal reviews or repeated reconciliation discrepancies.

Leading indicators can reveal deterioration before non-compliance occurs: growing case queues near due date, rising rejection-warning patterns, repeated interface interruptions, increasing aged discrepancies or declining completion of required EVDAS reviews.

A mature dashboard contains both. Lagging metrics measure realised compliance; leading indicators support intervention.

ICSR Reporting Metrics

On-time submission rate

A basic metric is:

number of reportable ICSRs submitted within the applicable regulatory timeframe ÷ total reportable ICSRs due in the period.

The denominator should be stable and auditable. Cases should not disappear from the denominator because of a late validity decision, vendor hand-off or technical issue.

Useful stratifications can include seriousness, product, country, intake source, vendor, case type or process step where these distinctions help identify causes. Stratification should be purposeful; excessive slicing can obscure the overall system picture.

Late-case profile

The absolute number and age of late cases can be more informative than a percentage alone. A 99.5% compliance rate may conceal a small number of very late cases with serious regulatory significance.

Instead of inventing fixed regulatory categories such as ">7 days late" or ">30 days late", organisations should define internal ageing bands that help prioritise investigation and escalation.

Near-due workload

A leading indicator can monitor open cases approaching their regulatory due date. The threshold is an internal operational decision and should reflect the organisation's workflow, volume and ability to recover from disruption.

Acknowledgement and Transmission Metrics

A submission attempt is not equivalent to accepted regulatory receipt. Useful indicators include:

• positive acknowledgement rate;
• rejection rate;
• unresolved rejection inventory;
• time from rejection to successful resubmission;
• repeated rejection codes or validation warnings; and
• missing acknowledgements requiring investigation.

The objective is not to force rejection rates to zero. Some exceptions will occur in complex systems. The governance question is whether the organisation detects them, understands patterns and prevents avoidable recurrence.

EMA's current electronic-reporting guidance also describes procedures for missing acknowledgements during quality-assurance testing and for system unavailability. Metrics should therefore distinguish organisation-controlled failures from documented EMA outages or agreed technical circumstances when evaluating performance.

Case Quality Metrics

Timeliness without quality can produce a misleading picture of compliance. EudraVigilance data support regulatory assessment and signal detection, so the content of submitted ICSRs matters as much as transmission speed.

Potential quality indicators include:

• proportion of sampled cases with material coding corrections;
• proportion of cases requiring follow-up because clinically important information was missing;
• duplicate-management discrepancies;
• product-identification errors;
• recurring E2B(R3) validation failures; and
• medically significant amendments after submission.

These should not be interpreted as universal regulatory KPI requirements. They are examples of how an organisation can monitor whether its case-processing controls remain effective.

A useful metric also distinguishes defect detection from defect prevention. An increase in detected coding issues can initially reflect stronger quality control rather than declining performance. Trend interpretation requires understanding the process that generated the number.

Reconciliation and Completeness Metrics

Reconciliation is a control used to test whether expected information has reached the intended system or process. The exact reconciliation model depends on the organisation's architecture and responsibilities.

Useful measures can include:

• planned reconciliation activities completed;
• open discrepancies by age and significance;
• repeated discrepancy categories;
• cases identified through reconciliation that should have been captured earlier; and
• time from discrepancy detection to resolution.

The frequency of reconciliation should be defined by the organisation according to risk and process design. EMA does not prescribe a universal monthly or quarterly reconciliation interval for every EudraVigilance-related process.

Signal-Management Metrics

For MAHs with EudraVigilance monitoring responsibilities, EVDAS and related signal-management activities create another important oversight layer. Metrics should focus on whether required reviews occur, whether outputs are scientifically evaluated and whether decisions are traceable.

Examples include:

• completion of planned EVDAS reviews;
• overdue review inventory;
• time from a statistical detection output to documented clinical review where applicable;
• validated signals awaiting assessment or regulatory action;
• repeated delays in signal governance; and
• signal records affected by later data-quality corrections.

A high number of detected statistical signals is not necessarily good or bad. It may reflect the product portfolio, data volume, threshold settings or methodological changes. The KPI should therefore support interpretation rather than reward volume.

Access and Registration Metrics

Because EudraVigilance access depends on registered organisations, named users and role approvals, governance metrics can help detect operational vulnerabilities.

Possible indicators include:

• active users by role;
• users without current operational need;
• pending access changes for joiners, movers and leavers;
• privileged EVDAS or level 2B access assignments;
• users whose required training or competency evidence is incomplete; and
• business-critical functions with only one trained or authorised user.

Periodic access-review completion can be useful, but the review interval is an internal control unless specified by an applicable requirement. The more meaningful outcome is whether inappropriate access is identified and corrected promptly.

Vendor and Partner Metrics

Vendor performance can be monitored using the same regulatory chain as internal work. A common weakness is to define metrics entirely around the vendor's contractual boundary.

For example, a vendor may report 100% on-time processing from vendor receipt to submission, while the MAH's true regulatory risk begins when any company representative or contracted channel first receives the case. Governance therefore needs end-to-end metrics where responsibility crosses organisational boundaries.

Useful vendor indicators can include:

• end-to-end reporting compliance;
• cases transferred late to or from the vendor;
• rejection and resubmission patterns;
• overdue follow-up or reconciliation items;
• recurring quality defects;
• significant system incidents; and
• overdue CAPAs or agreed corrective actions.

The objective is not to create as many vendor KPIs as possible. It is to detect whether outsourced activities threaten the pharmacovigilance system.

Deviations, CAPA and Trend Metrics

Deviation counts need context. A rising number may reflect deterioration, but it may also reflect improved detection and reporting culture. More informative measures can examine:

• severity and regulatory impact;
• recurrence of the same root cause;
• time to complete investigation;
• overdue CAPAs;
• effectiveness-check outcomes; and
• whether similar issues occur across products, affiliates or vendors.

A CAPA should not be considered effective merely because its due date was met. The metric should ask whether the failure mechanism has actually been controlled.

Building a QPPV Dashboard

A QPPV dashboard should be layered rather than overloaded. A practical model is:

Level 1 — system health

A small number of indicators showing material compliance and safety risk: serious/non-serious ICSR timeliness, unresolved major transmission failures, significant data-quality issues, overdue critical signal actions and major open deviations.

Level 2 — diagnostic trends

More detailed metrics explaining why Level 1 moved: intake-source delays, rejection codes, vendor performance, reconciliation discrepancies, access weaknesses or workload ageing.

Level 3 — transactional evidence

Underlying case lists, acknowledgement logs, signal records, deviation investigations and audit trails used when a metric requires investigation.

This hierarchy lets the QPPV maintain meaningful oversight without personally managing every transaction.

Interpreting Thresholds and Escalation

A threshold should exist because crossing it changes what the organisation does. Some thresholds come directly from regulation, such as an ICSR due date. Others are early-warning levels chosen internally to trigger review before non-compliance occurs.

Internal thresholds should be justified, documented and periodically reconsidered. A target that is so lenient that it never triggers action is ineffective; a target so strict that it generates constant noise can also fail because genuine risk becomes harder to distinguish.

Traffic-light dashboards are useful only when the decision logic behind red, amber and green is clear. The QPPV should be able to understand whether a red indicator represents an actual regulatory breach, an internal early-warning threshold or a data-quality anomaly.

Potential Failure Modes

The following are illustrative scenarios, not published inspection findings.

High compliance hides late source intake

The reporting dashboard starts the clock when the case enters the safety database, so affiliate delays are invisible. The KPI is mathematically correct but regulatorily misleading.

Averages hide serious outliers

An average rejection-resolution time appears acceptable while several high-priority serious cases remain unresolved. Distribution and aged-item review are needed alongside averages.

Targets become pseudo-regulations

An internal target such as 99% timeliness is described in procedures as an "EMA requirement". This confuses an organisational performance expectation with the underlying 15/90-day legal rules.

A metric has no owner or action

A dashboard shows increasing rejection rates for months, but nobody is accountable for investigating the trend. Measurement without decision rights does not constitute control.

Vendor metrics exclude MAH interfaces

The vendor performs perfectly inside the SLA, yet cases are delayed before transfer. End-to-end pharmacovigilance performance remains poor despite green vendor dashboards.

KPI definitions change without preserving comparability

A methodology change improves the apparent compliance rate, but historical data are not recalculated or clearly separated. Management may interpret a definitional change as genuine performance improvement.

Inspection and Management Review

An inspector can use metrics as an entry point rather than an endpoint. A red or repeatedly amber trend may be followed into source transactions, deviations, CAPAs and governance decisions. Conversely, an entirely green dashboard may itself prompt questions if known issues are absent from the reporting.

Potential review questions include:

• What regulatory obligation does this KPI help monitor?
• How are numerator and denominator defined?
• What is the source of the data and how is its completeness assured?
• Which thresholds are regulatory and which are internal?
• What happens when a threshold is exceeded?
• How are aged outliers made visible?
• Can the organisation explain important trends and discontinuities?
• Do vendor metrics measure the end-to-end process?
• How does the QPPV receive and challenge material EudraVigilance performance information?
• Can management demonstrate that recurring weaknesses led to effective action?

These are illustrative governance questions rather than a prescribed EMA dashboard specification.

Practical KPI Design Checklist

Before adopting a KPI, confirm that:

• it maps to a real pharmacovigilance risk or quality objective;
• the definition of receipt, due date, completion or defect is unambiguous;
• the numerator and denominator can be reproduced;
• exclusions are controlled and visible;
• the data source is reliable;
• the metric distinguishes regulatory requirements from internal targets;
• the threshold triggers a defined action;
• material outliers cannot be hidden by averages;
• trends can be compared over time despite methodology changes;
• relevant vendor and affiliate interfaces are included;
• the metric can be drilled down to supporting records; and
• the QPPV receives information at a level appropriate for system oversight.

Key Takeaways

EudraVigilance metrics are a means of governing the pharmacovigilance system, not an end in themselves. GVP Module I supports the use of performance indicators, but it does not mandate a single QPPV dashboard or universal internal thresholds.

The strongest KPI set combines regulatory compliance, leading indicators and diagnostic metrics. ICSR due dates remain regulatory obligations; percentages, ageing bands, rejection-resolution targets and traffic-light rules are normally organisation-defined controls.

A useful QPPV dashboard remains traceable to the underlying process. When a measure changes, the organisation should be able to explain which cases, signals, users, vendors or deviations produced the change and what action followed.

References

1. European Medicines Agency. GVP Module I – Pharmacovigilance systems and their quality systems. EMA/541760/2011. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-i-pharmacovigilance-systems-and-their-quality-systems_en.pdf
2. European Medicines Agency. GVP Module VI – Collection, management and submission of reports of suspected adverse reactions (Rev. 2). EMA/873138/2011 Rev. 2. https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vi-collection-management-submission-reports-suspected-adverse-reactions-medicinal-products-rev-2_en.pdf
3. European Medicines Agency. GVP Module IX – Signal management (Rev. 1) and Addendum I. Available from the EMA GVP collection. https://www.ema.europa.eu/en/human-regulatory-overview/post-authorisation/pharmacovigilance-post-authorisation/good-pharmacovigilance-practices-gvp
4. European Medicines Agency. EudraVigilance: electronic reporting. Current procedural guidance, support material and ICSR compliance information. https://www.ema.europa.eu/en/human-regulatory-overview/research-development/pharmacovigilance-research-development/eudravigilance/eudravigilance-electronic-reporting
5. European Medicines Agency. EudraVigilance system overview. https://www.ema.europa.eu/en/human-regulatory-overview/research-development/pharmacovigilance-research-development/eudravigilance/eudravigilance-system-overview
6. European Commission. Commission Implementing Regulation (EU) No 520/2012, as amended. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02012R0520-20260212

Regulatory Note

GVP Module I permits performance indicators to be used for continuous monitoring of pharmacovigilance activities, but it does not prescribe a universal EudraVigilance KPI set. Regulatory reporting timeframes and other legal obligations should be distinguished from company-defined targets, ageing bands, alert limits and escalation thresholds. KPI examples in this article are recommended operational practices unless an authoritative regulatory source specifically requires them.