EudraVigilance Metrics and KPIs for QPPVs

Introduction

A pharmacovigilance system cannot be effectively managed without meaningful metrics.

Although most organisations maintain some form of compliance reporting, many dashboards focus on operational workload rather than regulatory risk.

For QPPVs, the objective is not merely to understand what work was completed.

The objective is to understand whether the pharmacovigilance system remains compliant, controlled and capable of protecting patient safety.

EudraVigilance metrics provide visibility into the health of critical pharmacovigilance processes and can help identify emerging risks before they become inspection findings.

Why Metrics Matter

Metrics support:

• Compliance monitoring
• Trend analysis
• Resource planning
• Risk identification
• Vendor oversight
• Inspection readiness
• Management review

Without objective performance indicators, organisations may not recognise deteriorating compliance until a significant issue occurs.

Characteristics of a Good KPI

A useful pharmacovigilance KPI should be:

• Relevant
• Objective
• Actionable
• Repeatable
• Risk-focused

Metrics that cannot influence decision-making often add little value.

The best KPIs highlight emerging compliance risks early enough for intervention.

Reporting Compliance Metrics

Reporting compliance remains one of the most important EudraVigilance performance areas.

On-Time Reporting Rate

Measures the percentage of reports submitted within required timelines.

This KPI should be calculated at case-level and at product/MAH/vendor level to identify localized issues.

Example:

This metric is frequently reviewed by inspectors.

Late Reporting Rate

Tracks reports submitted after applicable deadlines.

A rising trend may indicate:

• Resource shortages
• Process bottlenecks
• Training issues
• System failures

Significant Reporting Delays

Monitor categories such as:

• Cases >7 days late
• Cases >15 days late
• Cases >30 days late

These indicators often reveal systemic weaknesses more effectively than average compliance percentages.

Practical implementation: - Define "case due date" consistently (e.g., date of first reportability determination or receipt of minimum criteria). - Use time-to-submission calculation aligned to GVP Module VI and Commission Implementing Regulation (EU) No 520/2012. - Exclude agreed and documented justified regulatory extensions or scientific exceptions; record rationale.

Acknowledgement Metrics

Submission alone does not demonstrate compliance.

Organisations should monitor acknowledgement outcomes.

Rejection Rate

Measures the percentage of reports rejected by EudraVigilance.

A high rejection rate may indicate:

• Coding issues
• Data quality problems
• Technical failures
• Validation weaknesses

Unresolved Rejections

Tracks rejected reports that remain open beyond defined timelines.

This metric can highlight compliance exposure.

Acknowledgement Review Completion

Measures whether acknowledgements are reviewed within established procedures.

Failure to review acknowledgements is a recurring inspection concern.

Practical implementation: - Maintain a log of EV acknowledgement messages with timestamps. - Link acknowledgement reviews to CAPA when patterns are noted (e.g., recurring rejection codes). - Use automated feeds where possible; retain screenshots and system extracts for audit trail.

Reconciliation Metrics

Reconciliation activities help ensure that reporting data remain complete and accurate.

Reconciliation Completion Rate

Measures completion of planned reconciliation activities.

Outstanding Discrepancies

Tracks unresolved differences identified during reconciliation.

Examples may include:

• Missing reports
• Duplicate reports
• Acceptance mismatches

Time to Resolution

Measures how quickly discrepancies are investigated and resolved.

Practical implementation: - Schedule reconciliations (monthly for high-risk products; quarterly otherwise). - Define reconciliation scope (EV vs safety database, EV acknowledgements vs submissions, literature vs safety database). - Maintain reconciliation templates with unique identifiers, status, responsible owner and evidence links.

Signal Management Metrics

Signal management metrics help demonstrate ongoing safety surveillance.

EVDAS Review Completion

Measures completion of scheduled EVDAS reviews.

For additional information see:

[[evdas-and-signal-detection]]

Signal Validation Timeliness

Tracks how quickly potential signals progress through validation activities.

Open Signal Inventory

Measures the number of signals currently under evaluation.

Trend monitoring may help identify resource constraints.

Signal Escalation Timeliness

Assesses whether significant safety concerns are escalated within expected timelines.

Practical implementation: - Document EVDAS review plans, outputs and escalation decisions. - Maintain signal logs with dates for detection, validation, assessment, and decision. - Link signals to risk management documents and regulatory submissions where required.

Medical Literature Monitoring Metrics

Literature surveillance processes may also be monitored.

Examples include:

• Literature review completion
• Case identification rates
• Literature case processing times
• MLM case review completion

For additional information see:

[[medical-literature-monitoring]]

Practical implementation: - Use a validated literature search strategy; retain search strings, results and screening logs. - Cross-reference literature-identified cases in EV/submission logs.

Access Governance Metrics

Access governance is often overlooked until inspections occur.

Useful indicators include:

Access Review Completion

Measures completion of scheduled access reviews.

Dormant Accounts

Tracks inactive users retaining access.

Training Compliance

Measures whether authorised users maintain required training status.

Segregation of Duties Exceptions

Identifies users holding potentially conflicting responsibilities.

Practical implementation: - Conduct formal access reviews at least annually, or more frequently for privileged roles. - Retain evidence: access review logs, remediation actions, approval signatures. - Link access changes to HR records and training completion.

Vendor Oversight Metrics

Outsourced activities should be monitored through defined KPIs.

Examples include:

• Reporting compliance
• Case processing timeliness
• Quality review outcomes
• Deviation rates
• CAPA completion

For additional information see:

[[vendor-oversight]]

Practical implementation: - Include KPI definitions and measurement methods in vendor contracts/SLA. - Require vendors to provide routine extracts and investigation evidence. - Perform trend analysis across vendors and include vendor performance in management review.

CAPA Metrics

CAPA effectiveness is often a key inspection focus.

Useful measures include:

Open CAPAs

Number of active CAPAs.

Overdue CAPAs

CAPAs exceeding planned completion dates.

Repeat Findings

Repeated observations may indicate ineffective CAPAs.

CAPA Effectiveness Rate

Measures whether implemented actions successfully resolved the underlying issue.

Practical implementation: - Retain CAPA records, evidence of completion and post-implementation effectiveness checks. - Link CAPAs to metrics used to identify the issue (e.g., rejection rate, late submissions). - Use root cause analysis documentation for critical CAPAs.

QPPV Dashboard Design

A QPPV dashboard should focus on risk rather than volume.

A practical dashboard often includes:

• Reporting compliance
• Rejection rates
• Reconciliation status
• Open signals
• Significant deviations
• Vendor performance
• CAPA status

The objective is to provide rapid visibility of compliance risks.

Design considerations: - Provide drill-down capability from aggregated KPI to individual case level. - Include trend lines and control limits. - Embed links to artefacts (SOPs, meeting minutes, deviation records). - Ensure single source of truth with RACI for metric ownership.

Trending and Thresholds

Individual metrics are useful.

Trends are often more useful.

Organisations should establish:

• Alert thresholds
• Escalation thresholds
• Review frequencies
• Management reporting expectations

Trend analysis helps identify gradual deterioration before compliance is affected.

Regulatory context: - Thresholds should be justified, documented and aligned with GVP requirements and internal risk tolerance. - Changes to thresholds should go through change control and be recorded for inspection.

Inspection Perspective

Inspectors frequently review metrics to understand how organisations monitor their pharmacovigilance systems.

Inspectors may assess:

• KPI definitions
• Governance review processes
• Escalation practices
• Trend analysis
• Management actions

Metrics are often viewed as evidence of system oversight.

Common expectations: - Clear definition of each KPI (numerator/denominator) - A rationale for targets - Documented ownership and data sources - Artefacts demonstrating ongoing review and corrective action

Common Metric Weaknesses

Observed weaknesses include:

• Excessive focus on workload
• Lack of risk indicators
• Missing trend analysis
• Poor escalation criteria
• Inconsistent reporting

A large dashboard does not necessarily indicate effective oversight.

Relevant metrics matter more than quantity.

Governance and Roles

Effective KPI governance requires:

• Defined owners for each KPI (with deputies)
• Documented SOPs describing calculation and use
• A governance forum (e.g., PV management review) with regular cadence
• Escalation pathways linked to thresholds (to QPPV, Safety Board, Executive)
• Integration with quality systems (audit, CAPA, change control)

Inspection relevance: - Inspectors expect governance records (meeting minutes, actions, evidence of follow up). - Demonstrate that metrics drive decisions (e.g., resource reallocation, CAPA initiation).

Practical KPI Framework for QPPVs

A simple EudraVigilance KPI framework may include:

This provides visibility across both operational and governance activities.

KPI Definitions and Inspection-Evidence Table

The table below provides inspection-ready KPI definitions with numerator/denominator, suggested targets, typical owners, data sources, reporting frequency and the artefacts an inspector expects to see. Use these as a baseline; adapt targets and frequencies to product risk, company size and regulatory commitments. Each KPI entry includes practical implementation guidance.

Notes on calculation and exclusions: - Ensure numerator and denominator definitions are in SOPs and align with regulatory definitions (e.g., "first reporter" rules, date of receipt). - Document any exclusions (e.g., duplicate reports, non-EU regulatory cases, invalidated cases) and retain rationale. - Use unique case identifiers when linking records across systems (safety DB, EV, vendor systems).

Implementation Practicalities

1. Single source of truth
2. Use a central reporting database or validated business intelligence (BI) tool that pulls data from the safety database, EV acknowledgement feeds and vendor reports.

3. Ensure data lineage is documented and auditable (who, when, how data are extracted and transformed).

4. SOPs and change control

5. Maintain SOPs describing KPI calculations, data sources, frequency and escalation pathways.

6. Changes to KPI definitions or thresholds must follow change control and be versioned.

7. Automation and validation

8. Automate KPI extraction where possible to reduce manual error.

9. Validate automated reports periodically (e.g., reconciliation between BI outputs and raw extracts).

10. Escalation pathways

11. Define clear escalation steps at alert and critical thresholds (e.g., inform QPPV within 24 hours for critical breaches).

12. Link escalation triggers to governance bodies (PV Quality Board, Executive Safety Committee).

13. Audit trail and artefacts

14. Store all artefacts in a controlled repository (PIM or document management system) with access for inspectors.

15. Artefacts should include extracts, SOPs, meeting minutes, CAPA evidence, vendor communications and change logs.

16. Training and competency

17. Train metric owners on calculation methods and inspection expectations.
18. Maintain competency records.

Regulatory Context and Inspection Relevance

• GVP Module I and Module VI underpin the requirement to monitor reporting systems and maintain quality systems. KPIs demonstrate ongoing capability and compliance.
• GVP Module III (Inspections) specifically references the expectation that organisations will have metrics to evidence oversight.
• Inspectors will typically:
• Verify KPI definitions match artefacts.
• Request raw extracts and traceability from KPI to individual case files.
• Review governance minutes showing use of KPIs in decision-making and CAPA initiation.
• Check evidence of escalation and regulator engagement where thresholds were breached.
• Be prepared to present:
• KPI calculation SOPs
• Raw data extracts for the period under review
• Dashboard screenshots with drill-down links
• Signed meeting minutes, CAPA records and correspondence

Inspection Readiness Checklist (for KPIs)

• Documented KPI definitions (numerator/denominator), targets and rationale
• SOP describing data sources and calculation method
• Evidence of periodic validation of KPI calculations
• Trend analyses and threshold definitions with documented triggers
• Governance records showing metric review and actions taken
• Linked artefacts (EV messages, reconciliation templates, CAPA files, vendor reports)
• Evidence of escalation to QPPV and management when thresholds breached

Key Takeaways

• Metrics help demonstrate pharmacovigilance system control.
• Reporting compliance remains a critical performance area.
• Acknowledgement and reconciliation metrics should not be overlooked.
• Signal management activities should be monitored through defined KPIs.
• Access governance and vendor oversight require objective measurement.
• QPPV dashboards should focus on compliance risk rather than activity volume.
• Inspectors frequently review metrics when assessing system effectiveness.

References

1. EMA Good Pharmacovigilance Practices (GVP) Module I – Pharmacovigilance Systems and Their Quality Systems.
2. EMA Good Pharmacovigilance Practices (GVP) Module III – Pharmacovigilance Inspections.
3. EMA Good Pharmacovigilance Practices (GVP) Module VI – Collection, Management and Submission of Reports of Suspected Adverse Reactions.
4. EMA Good Pharmacovigilance Practices (GVP) Module IX – Signal Management.
5. Regulation (EC) No 726/2004.
6. Directive 2001/83/EC.
7. Commission Implementing Regulation (EU) No 520/2012.