QPPV Metrics

Metrics help QPPVs maintain visibility over the performance, compliance and effectiveness of the pharmacovigilance system. Well-designed metrics, interpreted in the context of regulatory requirements and organisational risks, are a primary tool by which the QPPV exercises oversight.

Why Metrics Matter

Large pharmacovigilance systems produce high volumes of operational data. Metrics convert this data into actionable information that enables the QPPV to:

• Detect emerging compliance trends and quality deterioration early.
• Prioritise oversight and resource allocation using a risk-based approach.
• Provide evidence of active management to inspectors and to senior management.
• Monitor vendor performance and delegated activities against contractual and regulatory expectations.
• Link findings to corrective and preventive actions (CAPA) and evaluate CAPA effectiveness.

Regulatory guidance (for example EMA GVP Module I and ICH E2E principles) expects the QPPV to maintain oversight of the whole system and to ensure that appropriate metrics and management information are used to demonstrate that oversight.

QPPV Oversight: Regulatory Context

• European legislation requires a named QPPV with responsibility for establishing and maintaining the pharmacovigilance system (see Directive 2001/83/EC and related GVP).
• EMA GVP Module I describes the oversight expectations: regular review of safety data, timely reporting, risk management activities, system maintenance and ensuring adequate resources and governance.
• International guidance (including ICH E2E) supports adoption of quality systems and metrics to assure the integrity and timeliness of reporting.
• National competent authorities and inspectors will expect documented evidence that the QPPV actively reviews metrics, escalates issues, and signs off on remedial plans where necessary.

Principles of Metrics-Based Oversight

When implementing metrics for oversight, the QPPV should apply these principles:

• Fit-for-purpose: choose indicators that reflect regulatory obligations and material patient safety or compliance risk.
• Clear definitions: every metric must have an unambiguous definition, numerator, denominator, data sources, and calculation frequency.
• Data quality first: ensure data lineage, validation checks and reconciliation processes so metrics reflect reality.
• Risk-based thresholds: set thresholds based on risk tolerance, regulatory expectations and historical performance; define escalation triggers.
• Trend focus: prioritise trends and sustained deviation over single data points; use control charts and statistical process control where appropriate.
• Actionable: metrics should prompt specific actions (investigation, escalation, CAPA) not just reporting.
• Avoid perverse incentives: design metrics to encourage desired behaviour (quality and safety) rather than gaming.

Core Metric-Definition Template

To be inspection-ready and immediately implementable, every metric should be documented using a concise, standard template. Use the table below as the canonical metric-definition record stored in the metrics glossary (PSMF reference).

Store one filled template per metric in the metrics glossary, version-controlled and linked to the PSMF, SOPs, delegation log and relevant contracts.

Core Metric Definitions (Template Applied)

Below are representative, inspection-focused metric definitions for core QPPV oversight. Each entry uses the canonical template and includes brief implementation notes and inspection relevance.

1. Expedited Reporting Timeliness (15-day ICSRs)

Implementation notes: Use UTC timestamps and documented cutoff policy for receipt time. Automate extraction from safety database and reconcile weekly with inbox logs. Retain evidence of submission (transaction IDs, confirmation receipts) linked to the metric record.

Inspection relevance: Inspectors will request metric definitions, source data, reconciliation records and evidence of QPPV review of breaches and resulting CAPA.

2. Case Volume by Source

Implementation notes: Define source codes unambiguously and map affiliate source terms during onboarding. Automate alerts for spikes and annotate dashboards with contextual events (campaigns, regulatory actions).

Inspection relevance: Demonstrates capacity planning and ability to detect signal-generating reporting trends.

3. Case Backlog (Age-Based)

Implementation notes: Specify what "pending" means (e.g., awaiting triage, clinical review). Exclude planned holds (e.g., awaiting clinical trial database lock) if documented. Ensure audit trail supports age calculation.

Inspection relevance: Inspectors evaluate whether backlog metrics are tracked, trended and used to prevent reporting breaches.

4. Follow-up Rate (Expected Follow-ups Received)

Implementation notes: Define "expected follow-up" in SOPs (e.g., clinically relevant gaps). Track provenance of follow-up (investigator, HCP, patient, vendor). Include status codes and time windows.

Inspection relevance: Supports demonstration that completeness of information is tracked and that missing information is actively pursued.

5. Processing Cycle Time (Median and 95th Percentile)

Implementation notes: Define start and end events (system timestamps). Handle reworks and re-opened cases via policy to avoid skew. Use control charts to detect special cause variation.

Inspection relevance: Demonstrates ability to monitor and improve operational performance affecting compliance.

6. Reporting Compliance — Periodic Safety Reports (PSUR/PBRER)

Implementation notes: Maintain master submission calendar with time zones and national variations. Store submission evidence and correspondence in submission dossier.

Inspection relevance: Inspectors will review calendar, sign-offs, and late submissions plus mitigation.

7. CAPA Completion and Effectiveness

Implementation notes: Define verification criteria and evidence required for effectiveness. Maintain linkages between CAPA records and originating metric breaches.

Inspection relevance: Evidence that CAPA actions arising from metric triggers achieved intended outcomes is a frequent inspection focus.

8. Vendor SLA Compliance (Aggregate)

Implementation notes: Define KPI measurement rules in contracts. Include sample-based quality checks, and preserve raw evidence for audits.

Inspection relevance: Inspectors expect contractual metrics, evidence of monitoring, and records of escalation and resolution.

9. Data Quality — Coding/Error Rate

Implementation notes: Define sampling plan (e.g., 5–10% or minimum n per period) and sampling stratification by vendor/product. Use blinded review where feasible.

Inspection relevance: Shows QA oversight and maintenance of data integrity.

10. Audit and Inspection Findings

Implementation notes: Maintain an audit findings register with action owners, due dates, evidence attachments and effectiveness verification.

Inspection relevance: Demonstrates closure process, evidence trail, and QPPV engagement.

Practical Implementation Details for Metric Definitions

To ensure metric definitions are operational and inspection-ready, adopt the following implementation conventions:

• Timestamp policy: Define a single canonical timestamp for case receipt, submission, and other lifecycle events (e.g., UTC receipt timestamp from safety database). Document cut-off times for daily/weekly rolls.
• Inclusion/exclusion rules: For each metric record explicit inclusion and exclusion rules (duplicates, transfers, contested jurisdiction cases, non-reportable events). Where applicable include examples in the metric glossary to aid reproducibility.
• Automation and reconciliation: Automate metric extraction from the safety database and reconcile programmatically against independent sources (mailboxes, vendor reports, submission logs) on a defined cadence. Record reconciliation results.
• Rounding and percent calculations: Define rounding rules (e.g., two decimal places) and minimum denominator thresholds before percentages are reported (e.g., suppress percentage calculations if denominator < 5 and flag as low-volume).
• Sampling plans: For quality metrics, state the sampling methodology, sample size determination, stratification and how to increase sampling on poor performance.
• Version control and change history: Maintain a version-controlled metrics glossary with change rationale, effective date, and sign-off by metric owner and QA. Map changes to PSMF updates.
• Auditability: Link every reported metric to source data extracts and reconciliation evidence stored in the documentary evidence repository with access controls and retention policy aligned to regulatory expectations.
• Local variations: Where local reporting legislation diverges, maintain country-specific metric variants with their own thresholds and owners. Cross-reference these in the metrics glossary.

Governance, Roles and Accountability (Focus on Metrics)

Metrics are only effective with defined governance and accountability:

• Ownership hierarchy: Each metric must have a named owner (responsible for calculation and first-line action) and a sponsor (senior leader, typically PV Head or QPPV) who is accountable for the metric outcomes in governance forums.
• Review cadence: Define a schedule for metric review at operational (daily huddles), tactical (weekly operational review), strategic (monthly PV management) and executive (quarterly) meetings. The QPPV should present at strategic cadence and sign off material changes.
• Delegation and competence: Delegations for metric calculation and escalation must be recorded in the delegation log, with competency evidence for delegated staff; the QPPV retains ultimate oversight responsibility.
• Independent assurance: QA periodically audits metric integrity, sampling methods and use in oversight (at least annually and after major system changes). Results of QA reviews must be reported to the QPPV and governance committees.
• Documentation mapped to the PSMF: All metric definitions, thresholds, escalation pathways and evidence of QPPV involvement should be referenced in the PSMF and easily retrievable for inspection.
• Escalation matrix: Maintain a formal escalation matrix linked to metric thresholds with time-to-escalate and required artefacts (e.g., RCA, CAPA plan, meeting minutes). The QPPV must be included for all Red-level escalations.

Inspection Relevance — What to Provide Inspectors

Prepare inspection packs that map directly to metric templates and governance expectations:

• Metrics glossary export: a current, versioned file of all metric templates (name, purpose, numerator, denominator, frequency, owner, threshold, escalation).
• Source evidence: raw data extracts, reconciliation logs, and sample supporting documents for each metric.
• Governance artefacts: meeting minutes, dashboards presented to the QPPV, sign-offs, escalation emails and evidence of CAPA follow-through.
• Change history: records showing how metric definitions or thresholds changed, with rationale and QPPV/QA sign-off.
• Vendor evidence: SLA reports, vendor meeting minutes, KPI dashboards and any remediation evidence linked to vendor-led metrics.

Inspectors expect that metric-level definitions exist, are implemented consistently, and that the QPPV can demonstrate use of those metrics to detect, investigate and resolve issues.

Implementing the Metric Set — Practical Steps

1. Build the metrics glossary: Populate the canonical template for each core metric and store in the PSMF-linked repository.
2. Prioritise automation: Automate extraction and dashboarding for high-volume, time-critical metrics (e.g., expedited timeliness, backlog) and implement validation routines.
3. Implement reconciliation: Schedule automated reconciliations and manual spot checks; log reconciliations as part of the metric record.
4. Configure dashboards: Use role-based dashboards with drill-down for root cause analysis; annotate with events and CAPA links.
5. Define and test escalation: Run scenario-based tabletop exercises to test escalation pathways and timing; document results and lessons learned.
6. QA oversight: Plan independent QA review of metric calculations, sampling approaches and dashboards annually and after major changes.
7. Operationalise evidence retention: Ensure that evidence supporting metric values (submissions receipts, audit findings, vendor reports) is archived with metadata for inspection retrieval.

Interpretation and Avoiding Pitfalls

• Use multiple, corroborating metrics and qualitative evidence rather than single indicators.
• Interpret metrics within product lifecycle and event context.
• Avoid metrics that incentivise undesirable behaviour; use a balanced scorecard of leading and lagging indicators.
• Apply professional judgement: metrics inform action but do not replace QPPV decisions.

Recordkeeping and Evidence for Inspections

Do not rely on screenshots alone. Maintain:

• Raw data extracts with metadata and reconciliation logs.
• Versioned dashboards and exported reports (PDF/CSV) with timestamp and author.
• Meeting minutes with explicit QPPV attendance, decisions and sign-off.
• CAPA records linked to originating metric breaches and effectiveness test evidence.
• Delegation logs, competency records and vendor contracts with KPI clauses.

Role of the QPPV — Expanded Oversight Responsibilities

• Set oversight strategy and acceptable thresholds; retain the authority to adjust thresholds when justifiable and documented.
• Ensure metrics detect meaningful safety or compliance deficits and that escalation pathways are functional.
• Certify delegated activities are supervised and aligned with contractual metrics and regulatory commitments.
• Lead regulatory communications where systemic issues are uncovered via metrics.

Inspection Perspective — What Inspectors Look For

Inspectors typically seek:

• Clear metric definitions and documented thresholds in the metrics glossary.
• Evidence of QPPV engagement: dashboards, presentations, minutes and sign-offs.
• Root cause analyses and CAPA evidence following metric breaches.
• Vendor oversight evidence, including KPI reporting and escalation.
• Historical trend data to show how issues were detected and resolved.

Closing Summary

Metrics are a cornerstone of QPPV oversight when designed and used correctly. The metric-definition template provided here enables immediately implementable, inspection-ready documentation of core metrics. The QPPV must assure their validity, integrate them into governance, and use them with professional judgement to protect patient safety and maintain regulatory compliance.

Last reviewed: June 2026