Risk-Based Audit Planning in Pharmacovigilance

Introduction

Every pharmacovigilance (PV) system contains more auditable activities than available audit resources. Potential audit targets include case processing, signal management, aggregate reporting, risk management systems, vendors, affiliates, safety databases, QPPV oversight processes and quality systems. Attempting to audit everything equally is rarely practical. Modern PV audit programmes therefore rely upon risk-based planning to focus assurance where it matters most, demonstrate inspection readiness, and provide transparent governance rationales.

What Is Risk-Based Audit Planning?

Risk-based audit planning is the process of identifying, assessing and prioritising audit activity using a documented risk framework so that resources target areas with the highest potential impact on patient safety, regulatory compliance, data integrity and operational continuity. It shifts the question from "what can we audit?" to "what should we audit?" and requires documented evidence to support selection, frequency and scope decisions — information regulators routinely expect to review.

Why Risk-Based Planning Matters

Resource constraints (time, personnel, budget, expertise) make prioritisation unavoidable. Risk-based plans produce higher assurance per audit effort, strengthen inspection readiness, improve governance visibility and support a defensible audit posture during regulatory inspections. Regulators expect an auditable trail showing how risk data informed the audit plan; absence of this trail is a common inspection finding.

Regulatory Context and Inspection Relevance

Key regulatory references include: - EMA GVP Module IV — Pharmacovigilance Audits (inspection expectations for PV audits and audit programmes). - EMA GVP Module I — Quality systems and system-level requirements for PV. - EMA GVP Module III — Pharmacovigilance Inspections (how inspectors evaluate internal assurance and audit activities). - ICH Q9 — Quality Risk Management (principles for structured risk assessment). - Relevant regional legislation (e.g., Regulation (EC) No 726/2004; Directive 2001/83/EC).

Inspectors commonly request to see: - The audit universe and criteria for inclusion/exclusion. - The documented risk-scoring methodology and raw scoring records. - The annual audit plan with rationale and supporting evidence. - CAPA closure and effectiveness measures arising from prior audits. - Audit scheduling decisions for high-risk vendors and affiliates.

An inspection-ready audit programme not only has a defensible plan but also the linked documentation (scorecards, source data, governance approvals, and follow-up evidence).

Building the Audit Universe

The audit universe is the comprehensive list of auditable PV activities. Typical categories: - Core PV activities: ICSR intake/triage, case processing, medical review, expedited reporting, case quality control. - Signal management: detection, evaluation, prioritisation, action. - Aggregate reporting: PSURs/PBRERs, DSURs, periodic reporting process controls. - Governance: QPPV oversight, SOP control, training, management review. - Vendor/outsourced services: safety database vendors, call centres, laboratories, CROs. - Data systems and integrity: E2B transmissions, database migrations, access controls. - Quality system processes: CAPA, deviations, change control, internal audits.

Each auditable unit should be described, assigned an owner, and be capable of being risk-assessed.

Risk Assessment Frameworks and Scoring Models

A consistent, documented scoring model is essential. Typical factors include patient safety impact, regulatory impact, data integrity, complexity, change exposure, and historical performance. Models can be additive (weighted sum), multiplicative, or hybrid.

Recommended model elements: - Define factor descriptions and scoring anchors (e.g., 1 = negligible, 5 = critical). - Define weighting for each factor based on organisational priorities (e.g., patient safety may be higher weight than operational disruption). - Document how composite scores are calculated and threshold bands for Low/Medium/High/Critical risk. - Capture source data and evidence used for each factor (e.g., KPI reports, inspection findings, change logs).

Consistency and traceability are critical for inspection readiness.

Example Risk Scoring Model (Worked Example)

Below is a worked example showing step-by-step risk scoring, calculation and resulting audit priority.

Scoring factors and weights: - Patient Safety Impact (PS) — weight 35% - Regulatory Impact (RI) — weight 25% - Data Integrity / Availability (DI) — weight 15% - Complexity / Volume (CV) — weight 10% - Change Exposure (CE) — weight 10% - Historical Performance (HP) — weight 5%

Each factor scored 1–5 (1 = low risk, 5 = highest risk).

Calculation: Weighted Sum Score = (PS0.35 + RI0.25 + DI0.15 + CV0.10 + CE0.10 + HP0.05). Normalize to 1–5 scale (same range).

Thresholds: - 4.25–5.00 = Critical - 3.50–4.24 = High - 2.50–3.49 = Medium - 1.00–2.49 = Low

Worked example audit universe subset (three auditable units):

1. ICSR Case Processing (Global)
2. PS = 5 (direct impact on patient safety & timeliness of expedited reporting)
3. RI = 5 (regulatory reporting obligations)
4. DI = 4 (data completeness & accuracy critical)
5. CV = 5 (high volume)
6. CE = 3 (system upgrade planned in 6 months)

7. HP = 3 (some repeated minor findings) Weighted Sum = (50.35) + (50.25) + (40.15) + (50.10) + (30.10) + (30.05) = 1.75 + 1.25 + 0.60 + 0.50 + 0.30 + 0.15 = 4.55 → Critical

8. Vendor — Safety Database Provider (Vendor A)

9. PS = 4 (affects case processing)
10. RI = 4 (data transfers, regulatory reporting)
11. DI = 5 (system is source of truth; integrity critical)
12. CV = 4 (all corporate cases)
13. CE = 2 (no major pending changes)

14. HP = 4 (recent major audit findings, partially remediated) Weighted Sum = (40.35) + (40.25) + (50.15) + (40.10) + (20.10) + (40.05) = 1.40 + 1.00 + 0.75 + 0.40 + 0.20 + 0.20 = 3.95 → High

15. Literature Surveillance (Global)

16. PS = 3 (supports signal detection but lower direct reporting impact)
17. RI = 3 (relevant for safety summaries)
18. DI = 3 (accuracy important but lower risk)
19. CV = 2 (moderate volume)
20. CE = 4 (new external publications and automated tools being integrated)
21. HP = 2 (no recent findings) Weighted Sum = (30.35) + (30.25) + (30.15) + (20.10) + (40.10) + (20.05) = 1.05 + 0.75 + 0.45 + 0.20 + 0.40 + 0.10 = 2.95 → Medium

Resulting prioritisation: - ICSR Case Processing — Critical (top audit priority; schedule immediate comprehensive audit) - Vendor A (Safety Database) — High (vendor audit prioritised this year with follow-up) - Literature Surveillance — Medium (monitor and consider targeted review aligned with tool integration)

Documentation note for inspection: retain scoring worksheets, source evidence (KPI reports, vendor audit reports, change logs), rationale for weight selection, and governance approval for thresholds.

Translating Scores into Audit Frequency and Scope

Suggested mapping: - Critical: Comprehensive audit within 6 months; follow-up within 3–6 months; consider continuous monitoring or quarterly KPI reviews. - High: Full audit within 12 months; targeted interim checks or sampling. - Medium: Targeted audits on a 12–36 month cycle; desktop reviews or thematic audits. - Low: Desk reviews, process monitoring, and longer audit cycles (36+ months) unless risk increases.

Document the relationship between score bands and intended assurance level in the audit programme to demonstrate consistent application.

Inspection-Ready Audit Planning Checklist

The following checklist is designed for immediate use when preparing an audit plan and for inspection readiness evidence. For each item, maintain the specified documentary evidence and be prepared to explain during an inspection.

Inspection-Ready Audit Planning Checklist - Audit Universe - Item: Defined and current audit universe. - Evidence: Audit universe master list (date-stamped), auditable unit descriptions, owners. - Inspection Relevance: Inspectors will verify completeness and inclusion rationale.

• Risk Assessment Methodology
• Item: Documented scoring model, factor definitions, weights and thresholds.
• Evidence: Risk scoring SOP / guidance; scoring matrix document.

• Inspection Relevance: Inspectors ask how risk was assessed and thresholds set.

• Source Data and Inputs

• Item: Evidence that inputs were used (KPI trends, inspection findings, CAPA status, changes, volumes).
• Evidence: KPI reports, inspection findings summary, CAPA trackers, change logs, organisational restructure notes.

• Inspection Relevance: Inspectors check that decisions are evidence-based.

• Scoring Records and Worksheets

• Item: Completed scoring worksheets for each auditable unit.
• Evidence: Signed/dated scoring tables, calculation spreadsheets, supporting comments for each factor.

• Inspection Relevance: Inspectors expect to see how each score was derived.

• Audit Plan (Annual / Multi-year)

• Item: Documented plan showing scheduled audits, scope, rationale and resource allocation.
• Evidence: Annual audit plan with risk-based justification, audit charters, resource assignment.

• Inspection Relevance: Inspectors review rationale for selection, timing and frequency.

• Prioritisation Rationale

• Item: Clear mapping from risk score to audit priority and frequency.
• Evidence: Policy section or decision log linking scores to scheduled activity.

• Inspection Relevance: Inspectors examine consistency of prioritisation.

• Governance Approvals

• Item: Evidence of governance review and approval (QM, QPPV, Senior Management).
• Evidence: Approval emails, minutes of Audit Committee or Governance Board, sign-off forms.

• Inspection Relevance: Inspectors look for governance oversight and accountable owners.

• Resource and Capability Assessment

• Item: Evidence that the audit team has required skills or plan to source them.
• Evidence: Auditor CVs, training records, outsourced audit contracts.

• Inspection Relevance: Inspectors evaluate auditor independence and competence.

• Vendor Audit Strategy

• Item: Vendor risk classification and audit schedule.
• Evidence: Vendor risk register, SLAs, vendor audit reports, corrective action records.

• Inspection Relevance: Inspectors focus on critical outsourced processes and oversight.

• Follow-up and CAPA Tracking

• Item: Process for tracking findings, CAPAs and effectiveness checks.
• Evidence: CAPA tracker, closure evidence, effectiveness assessments, re-audit plans.

• Inspection Relevance: Inspectors check closure and effectiveness rather than only closure status.

• Dynamic Review and Trigger Mechanisms

• Item: Triggers for ad-hoc audits (e.g., new product, system migration, regulatory inspection, major findings).
• Evidence: Trigger log, evidence of triggered audits, change management records.

• Inspection Relevance: Inspectors wish to see that the plan adapts to emerging risks.

• Audit Documentation and Records Management

• Item: Centralised audit repository and retention policy.
• Evidence: Audit files, SOP for audit documentation, archive evidence.

• Inspection Relevance: Inspectors review audit records for traceability.

• Communication and Escalation Pathways

• Item: Clear lines for escalation of critical findings to QPPV and senior management.
• Evidence: Escalation procedure, recent escalation examples, meeting minutes.

• Inspection Relevance: Inspectors check escalation timeliness and senior management involvement.

• Continuous Improvement Metrics

• Item: KPIs for audit programme performance (coverage, findings trends, CAPA timeliness).
• Evidence: Audit programme KPI dashboard, trend analyses, programme review minutes.
• Inspection Relevance: Inspectors assess whether the audit programme demonstrates improvement over time.

Use the checklist as a living template. During inspections, auditors commonly request to see the items above; having these artifacts compiled reduces inspection time and exposure.

Sample Audit Calendar (Practical Implementation Example)

Below is a sample audit calendar for a medium-sized PV organisation using the risk-scoring outputs from the worked example. The calendar demonstrates allocation of audit resources across the year, prioritisation of critical items, vendor coverage and governance activities.

Assumptions: - Total audit-days available (PV QA): 120 days for the year. - Additional vendor audit budget: 40 days (external + internal). - Critical audits require 10–15 audit-days; full vendor audit 8–12 days; targeted/desktop review 2–5 days.

Sample Calendar (Quarterly view):

Q1: - January - ICSR Case Processing (Comprehensive) — 15 days (Critical) — scope: end-to-end global case intake to submission, QC sampling across regions. - Audit Planning & Governance Meeting — 1 day (audit plan sign-off, resource assignment) - February - Vendor A — Safety Database (On-site / Remote hybrid) — 10 days (High) — scope: data integrity, change control, system interfaces, business continuity. - March - CAS (Corrective Action Status) Review (Desktop) — 3 days — scope: effectiveness of CAPAs from prior year critical audits. - Literature Surveillance — Targeted Review (Desktop) — 3 days (Medium) — focus on tool integration controls.

Q2: - April - Aggregate Reporting (PBRER Preparation Process) — 8 days (High) — scope: data collation, review timelines, sign-off controls. - May - Affiliate PV Oversight (Regional) — 6 days (High) — scope: local case processing, reporting compliance, local QPPV engagement. - June - Training & Competence Review — 2 days (Desktop) — scope: PV training completion rates, competence records.

Q3: - July - Signal Management (Process & Case Studies) — 8 days (High) — scope: detection algorithms, triage, documentation of actions. - August - Vendor — Call Centre (Case Intake) — 8 days (High) — scope: call handling, quality monitoring, escalation process. - September - Data Migration Readiness (if scheduled) — 6 days (Targeted) — scope: data mapping, reconciliation plans.

Q4: - October - Safety Database Re-audit — 4 days (Follow-up targeted) — scope: remediation verification, data integrity spot-checks. - November - Internal Audit of PV Quality System — 6 days (Comprehensive) — scope: CAPA effectiveness, audit scheduling, document control. - December - Year-End Programme Review and Audit Calendar Draft for Next Year — 3 days — governance meeting with QPPV and Senior Management.

Resource summary: - PV QA days used: approx. 120 days (including desk reviews and governance time) - Vendor audit days used: 40 days - Reserve: 10–15 days contingency for triggered ad-hoc audits (inspections, critical findings, migrations)

Inspection relevance: maintain a cross-reference matrix that maps each calendar audit to its underlying risk score, scoring worksheet, supporting evidence and governance approval. Inspectors will request both the plan and the trail showing why each audit was scheduled when it was.

Governance and Oversight

Robust governance ensures the audit programme is both effective and defensible:

Roles and responsibilities: - Audit Owner / Head of QA: maintains audit universe, ensures methodology is applied consistently, oversees resource allocation. - QPPV: provides clinical and regulatory judgement, input on high-priority risks and emergent safety issues. - Senior Management / Audit Committee: approves the annual audit plan, monitors programme KPIs and CAPA effectiveness, ensures independence and resourcing. - Auditors: execute audits with appropriate competence, document observations, and support CAPA development and verification. - Vendor Management: maintains vendor risk register and supports audit logistics and follow-up.

Governance practices for inspection readiness: - Annual sign-off of the audit plan with documented rationale and recorded meeting minutes. - Periodic (e.g., quarterly) review of the audit universe and triggered audits log. - Transparent escalation routes for critical findings (immediate report to QPPV and senior management). - Regular reporting of audit KPIs (audit coverage vs universe, time-to-CAPA closure, reoccurrence rates). - Retention of all audit planning and scoring artefacts in a secure, searchable repository for inspection access.

Inspectors expect to see both operational and governance artefacts demonstrating that audit decisions are not arbitrary, but the product of a structured, evidence-based process with oversight.

Follow-up: CAPA, Effectiveness Checks and Continuous Monitoring

A risk-based plan is incomplete without a structured follow-up approach: - CAPA assignment within a defined timeframe, with responsible owner and target completion dates. - Verification of CAPA implementation and measurement of effectiveness using pre-defined criteria. - Link findings and CAPAs back into the risk scoring process (e.g., improvement in Historical Performance factor). - Use KPIs and trend analysis to adjust audit priority and frequency dynamically.

Documented evidence of CAPA effectiveness is a frequent focus for inspectors.

Common Pitfalls and How Inspection Teams View Them

Inspectors commonly identify the following weaknesses: - Lack of traceability between risk assessment inputs and audit selection. - Undocumented or inconsistent weighting/scoring methods. - No contingency for emergent risks (static calendar with no triggers). - Incomplete vendor oversight documentation (missing SLAs, no critical vendor audit). - Governance absent or weak (no approvals, no escalation evidence).

Address these proactively by preserving scoring worksheets, meeting minutes, approvals and CAPA verification artefacts.

Characteristics of a Mature Programme (Inspection Perspective)

Inspectors consider the following features favourable: - A documented, consistently applied risk model with evidence for scores. - A dynamic audit universe with triggers and evidence of adaptation. - Clear governance with QPPV visibility on priorities and rapid escalation mechanisms. - An auditable trail linking risk assessment → audit plan → audit execution → CAPA → effectiveness checks. - Appropriate auditor competence and demonstrable independence (including vendor audits).

Key Takeaways

• Risk-based audit planning aligns audit resources with areas of highest patient safety, regulatory and operational impact.
• Use a documented scoring model with defined weights, thresholds and source evidence; retain scoring worksheets and the inputs for inspection.
• Translate scores to frequencies and scope, and publish an audit calendar that includes contingency for triggered audits.
• Maintain governance oversight with explicit approvals, escalation paths and KPI reporting.
• Prepare an inspection-ready file containing the audit universe, scoring records, the annual audit plan, audit reports, CAPA trackers and evidence of CAPA effectiveness.
• Dynamic review of risk and adaptation of the audit plan is essential to remain aligned with evolving product portfolios, vendors and regulatory expectations.

References

1. EMA Good Pharmacovigilance Practices (GVP) Module IV – Pharmacovigilance Audits.
2. EMA Good Pharmacovigilance Practices (GVP) Module I – Pharmacovigilance Systems and Their Quality Systems.
3. EMA Good Pharmacovigilance Practices (GVP) Module III – Pharmacovigilance Inspections.
4. Regulation (EC) No 726/2004.
5. Directive 2001/83/EC.
6. Commission Implementing Regulation (EU) No 520/2012.
7. ICH Q9 Quality Risk Management.
8. ICH E2E Pharmacovigilance Planning.