Vendor Risk Assessment in Pharmacovigilance

Table of Contents

• Introduction
• Why Vendor Risk Assessment Matters
• The Regulatory Perspective
• What Is Vendor Risk?
• Risk-Based Oversight
• Common Vendor Risk Factors
• A Practical Risk Classification Model
• Critical Vendors
• Building a Risk Scoring Model
• Risk Assessment During Vendor Selection and Operations
• Risk Assessment and Audit Frequency
• Risk Assessment and KPIs
• Risk Assessment and QPPV Oversight
• Common Risk Assessment Mistakes
• Inspection Perspective
• Characteristics of Mature Vendor Risk Management
• Key Takeaways
• Appendix: Inspection‑ready materials — checklist, worked example, and template risk register entry
• A. Risk assessment checklist (illustrative operational model)
• Administrative and core documents
• Risk assessment artefacts
• Quality and performance evidence
• Operational controls and evidence
• Governance and escalation
• Inspection-specific evidence and navigation aids
• Practical inspection presentation notes
• Why each checklist item matters to inspectors
• B. Worked scoring example with thresholds (including implementation details and governance rules)
• Model assumptions and governance rules
• Inspection relevance and evidence
• Notes on practical implementation
• C. Template risk register entry (illustrative operational model)
• Sample filled entry — Vendor A (Global Case Processing Vendor)
• Governance and operational guidance for the register
• Inspection relevance
• D. Implementation checklist and governance controls (practical steps)
• Regulatory context and mapping
• Closing note on inspection preparedness
• References
• Regulatory Note

Introduction

Not all vendors create the same level of pharmacovigilance risk.

A vendor providing translation services presents a different risk profile than a vendor responsible for expedited safety reporting. Despite this reality, some organisations apply identical oversight approaches to all vendors. This frequently results in:

• Excessive oversight of low-risk vendors
• Insufficient oversight of high-risk vendors
• Inefficient use of resources
• Inspection findings

Risk assessment helps solve this problem by enabling organisations to allocate oversight effort proportionally and maintain effective control of outsourced activities.

Why Vendor Risk Assessment Matters

Modern pharmacovigilance systems rely heavily on outsourcing. Examples include:

• Case processing
• Literature surveillance
• Aggregate reporting
• Safety database hosting
• Medical information services
• Signal management support

Each outsourced activity withdraws a degree of direct organisational control. The objective of vendor risk assessment is not to eliminate risk but to understand, prioritise and control it — thereby focusing attention and resources where they reduce the greatest regulatory and patient-safety exposures.

The Regulatory Perspective

Regulators expect a risk-based approach to pharmacovigilance governance, inclusive of vendor oversight. Inspectors commonly ask for evidence of:

• How vendors are classified and why
• Criteria used to establish audit frequency and oversight intensity
• Documentation demonstrating that oversight is proportional to assessed risk
• Identification and management of critical vendors

Absent a structured, documented risk model and supporting evidence, organisations struggle to provide consistent answers and may receive inspection findings. Key references include EMA GVP Module I (quality systems), GVP Module III (inspections), ICH Q9 (Quality Risk Management), and regional regulations governing pharmacovigilance obligations.

What Is Vendor Risk?

Vendor risk refers to the possibility a vendor's actions (or failures) could negatively affect:

• Patient safety
• Regulatory compliance
• Data integrity
• Operational continuity
• Product quality
• Inspection outcomes

Risk is a function of both the likelihood of failure and the consequence of that failure. Effective vendor risk assessments make both dimensions explicit and defensible.

Risk-Based Oversight

Oversight effort should be proportional to risk. Typical responses by category:

• Low risk: Basic oversight (periodic review, annual KPI review)
• Medium risk: Periodic on-site/remote assessments, defined KPIs, contractually mandated reporting
• High risk: Regular audits, enhanced KPIs, governance meetings with senior stakeholders
• Critical risk: Frequent audits or audits-as-required, QPPV-level visibility, contingency planning, senior management escalation

Risk assessment defines which combination of controls, frequency, and escalation is required and documents the rationale for inspection.

Common Vendor Risk Factors

Typical risk factors used for classification include:

• Regulatory Impact: Potential to cause regulatory non-compliance (e.g., expedited reporting)
• Patient Safety Impact: Direct effect on identification or management of safety events (e.g., signal management)
• Data Integrity Impact: Responsibility for safety data or database functionality
• Business Continuity Impact: Single-source dependency, ability to replace vendor
• Geographic/Regulatory Scope: Number and type of regions dependent on vendor performance
• Operational Complexity: Scope, variation, and change frequency of the activity
• Subcontracting and Resilience: Use of downstream vendors and their oversight
• Historical Performance: Audit outcomes, CAPA history, KPI trends

These factors must be documented and scored consistently.

A Practical Risk Classification Model

Organisations commonly adopt simple, repeatable classification scales. A typical approach uses per-factor scoring (e.g., 1–5), optional weighting for high-impact factors (such as Regulatory Impact), and sum or weighted-sum totals mapped to categorical thresholds (Low / Medium / High / Critical). Consistency, documented rationale, and change control are more important than the exact numeric scheme.

Critical Vendors

Critical vendors are those whose failure would cause missed reporting, patient-safety risk, major data loss, or regulatory non-compliance. These vendors require:

• Enhanced governance (regular steering committee attention)
• Enhanced metrics (near real-time KPIs where appropriate)
• Enhanced audit activity (more frequent or deeper audits)
• Contractual controls (clear service levels, penalties, transition arrangements)
• Contingency plans and tested business continuity arrangements
• Direct QPPV awareness and documented escalation paths

Inspections frequently probe whether critical vendors have been identified and whether contingency plans are credible and tested.

Building a Risk Scoring Model

Elements of a practical scoring model:

• Define scoring scale (e.g., 1–5) and what each point means
• Define factors (Regulatory Impact, Patient Safety, Data Integrity, Business Continuity, Complexity)
• Optionally apply weights to factors of greater regulatory importance
• Define thresholds mapping totals to risk categories
• Document residual risk calculations (initial score minus effect of controls)
• Require narrative rationale for scores and links to objective evidence
• Store scores with version control and sign-off

The model must be validated, simple enough to be applied consistently, and auditable.

Risk Assessment During Vendor Selection and Operations

Risk assessment starts in vendor selection and continues through the contract lifecycle. Key moments for reassessment include:

• Contract initiation
• Scope changes
• Technology/platform changes
• Poor performance or CAPA events
• Regulatory changes or inspection findings
• Periodic scheduled reviews (e.g., annually or biannually for high-risk vendors)

The vendor file should contain evidence of initial and subsequent assessments.

Risk Assessment and Audit Frequency

Risk classification should be the primary determinant of audit frequency, within the constraints of resources and risk appetite. Typical models map categories to audit cadence but allow for exceptions driven by strategic considerations or recent performance.

Risk Assessment and KPIs

KPIs and quality metrics should be risk-proportionate: the higher the assessed risk, the more granular and frequent the monitoring should be. Examples of KPIs include case processing timeliness, processing quality metrics, signal detection metrics, and CAPA closure rates.

Risk Assessment and QPPV Oversight

The QPPV must have visibility over critical vendor risks and significant issues likely to affect pharmacovigilance obligations. Mechanisms to ensure this include scheduled reporting, direct access to vendor metrics, and participation in escalation meetings.

Common Risk Assessment Mistakes

Common deficiencies include:

• One-time assessments without periodic review
• Uniform oversight regardless of risk
• Overly complex scoring schemes that reduce repeatability
• Poor documentation and weak evidence trails
• Failure to recognise organisational dependency on single-source vendors

These mistakes are frequent triggers for inspection findings.

Inspection Perspective

Inspectors commonly request evidence that:

• Vendors were classified using a documented methodology
• The oversight intensity is justified by the assessed risk
• Critical vendors are identified and governed appropriately
• Audits, CAPAs and governance actions are documented, tracked and closed
• Contingency plans exist and have been tested for critical vendors

Prepare to demonstrate the entire lifecycle for a sample of vendors: from selection/risk assessment to ongoing monitoring, audit activity, CAPAs, and closure.

Characteristics of Mature Vendor Risk Management

Mature programmes demonstrate:

• Explicit, structured classification criteria
• Documented rationale and evidence for each assessment
• Periodic reassessment and change control
• Risk-based oversight implemented and resourced
• Integration of vendor risk information into enterprise governance and inspection readiness

These attributes form the basis for defensible, inspection-ready vendor oversight.

Key Takeaways

• Vendor risk assessment underpins proportionate oversight.
• Not all vendors should receive the same governance intensity.
• Identify critical vendors and document the rationale.
• Link risk assessment outcomes to audits, KPIs, contracts and contingency plans.
• Maintain an auditable trail: records, minutes, metrics, CAPAs and reviews.

• Regulatory expectations require both a method and demonstrable implementation.

• EMA Good Pharmacovigilance Practices (GVP) Module III – Pharmacovigilance Inspections.

• EMA Good Pharmacovigilance Practices (GVP) Module II – Pharmacovigilance System Master File.
• Regulation (EC) No 726/2004.
• Directive 2001/83/EC.
• Commission Implementing Regulation (EU) No 520/2012.
• ICH Q9 Quality Risk Management.
• ICH E2E Pharmacovigilance Planning.

Appendix: Inspection‑ready materials — checklist, worked example, and template risk register entry

This appendix provides inspection‑ready artefacts you can adopt or adapt: (A) a comprehensive risk assessment checklist to evidence compliance and demonstrate readiness; (B) a worked scoring example with thresholds and resulting oversight actions; (C) a template risk register entry (blank and a filled sample) ready for inclusion in your vendor risk register. Each element includes implementation details, governance expectations, inspection relevance and regulatory mapping.

A. Risk assessment checklist (illustrative operational model)

Use this checklist to compile a vendor file for inspection. Items should be present, version-controlled, and easily retrievable.

Administrative and core documents

• Vendor identification: legal entity name, trading names, address(es), DUNS/Tax ID. (GVP I)
• Scope of services: detailed statement of work (SOW) / description of pharmacovigilance services and deliverables. (GVP II)
• Contract and amendments: full executed contract, annexes, SLAs, change control history. Include data protection and confidentiality clauses. (GVP I)
• Master Service Agreement (MSA) and subcontracts: evidence of approval and oversight of subcontractors. (GVP III)

Risk assessment artefacts

• Initial risk assessment: completed scoring form, factor-level scores, and narrative rationale. (ICH Q9)
• Residual risk assessment: controls considered, adjusted scores and explanation.
• Version history and sign-off: names, roles (Vendor Manager, QA, QPPV), dates. (GVP I)

Quality and performance evidence

• KPIs and monitoring reports: historical trend data, thresholds, escalation records.
• Audit history: audit reports, non-conformances, corrective and preventive actions (CAPAs), timelines and closure evidence. (GVP III)
• CAPA documentation: root cause analyses, action owners, verification of effectiveness, and closure evidence.

Operational controls and evidence

• Standard operating procedures (SOPs) referencing vendor oversight and interactions.
• Training records for vendor staff where required, and evidence of staff qualifications.
• Business continuity/ disaster recovery plans and test results (RTO/RPO evidence).
• Data integrity controls: validation reports for systems, data migration evidence, access controls.
• Systems architecture/topology for hosted solutions and hosting provider contracts.

Governance and escalation

• Governance meeting minutes or steering committee records that include vendor performance discussion.
• QPPV and senior management briefings where vendor issues have been escalated.
• Evidence of contractual remedies applied (if any) and implementation.

Inspection-specific evidence and navigation aids

• Index page for vendor file with links to documents and master PDF or binder with tabbed sections.
• A 1-page executive summary of the vendor risk assessment showing classification and oversight plan.
• A searchable export of the risk register entry for the vendor (showing historical changes).
• Evidence of periodic reassessment (calendar entries, review dates).
• Copies of corrective actions taken in response to inspection-related findings (if vendor was involved).

Practical inspection presentation notes

• Provide a single PDF package containing: executive summary, risk score spreadsheet, contract, last audit report and last three KPI reports.
• Ensure documents have dates and sign-offs. Where records are electronic, provide screenshots showing metadata (created/modified dates).
• Prepare a short scripted narrative linking risk factors to oversight activities and evidence.
• Identify a contact list (Vendor Manager, QA Lead, QPPV) and documented delegation of responsibilities.

Why each checklist item matters to inspectors

• Demonstrates a lifecycle approach (selection → monitoring → action) and traceability.
• Links assessment outcomes to controls and governance decisions.
• Shows that risk-based decisions are not hypothetical but implemented and monitored.

B. Worked scoring example with thresholds (including implementation details and governance rules)

Model assumptions and governance rules

• Scoring scale: 1 (lowest) to 5 (highest) for each factor.
• Factors used (standard set): Regulatory Impact (weight 1.5), Patient Safety Impact (weight 1.5), Data Integrity Impact (weight 1.2), Business Continuity Impact (weight 1.0), Operational Complexity (weight 1.0). Weights reflect organisational emphasis on regulatory and patient-safety consequences and can be adjusted in the governance document.
• Initial score = sum(weight × raw score) across factors.
• Controls evaluation: for each factor, list key controls (contractual SLA, validation evidence, redundancy, certified processes). Estimate control effectiveness qualitatively (High, Medium, Low) and apply control reduction factor (High = -20%, Medium = -10%, Low = 0%) to compute residual score.
• Thresholds (example):
• Low risk: residual score < 6
• Medium risk: 6 ≤ residual score < 10
• High risk: 10 ≤ residual score < 14
• Critical risk: residual score ≥ 14
• Governance triggers:
• Residual score ≥ 14: immediate QPPV notification, executive escalation, mandatory audit within 90 days and documented contingency plan.
• Residual score 10–13.9: scheduled audit within 12 months and monthly KPI review.
• Residual score 6–9.9: periodic review (annual) and KPIs quarterly.
• Residual score < 6: annual check and spot-monitoring.

Worked example: Vendor A — Global Case Processing Vendor (safety case intake and initial processing for multiple EU subsidiaries)

Step 1: Factor scoring (raw scores 1–5) - Regulatory Impact: 5 (vendor performs expedited reporting and sends ICSRs to authorities) - Patient Safety Impact: 5 (initial triage and seriousness assessment) - Data Integrity Impact: 4 (primary case entry and database hosting) - Business Continuity Impact: 5 (sole provider for several products) - Operational Complexity: 4 (multiple languages, regional variations)

Step 2: Apply weights and compute initial score - Regulatory Impact: 5 × 1.5 = 7.5 - Patient Safety Impact: 5 × 1.5 = 7.5 - Data Integrity Impact: 4 × 1.2 = 4.8 - Business Continuity Impact: 5 × 1.0 = 5.0 - Operational Complexity: 4 × 1.0 = 4.0 - Initial aggregated score = 7.5 + 7.5 + 4.8 + 5.0 + 4.0 = 28.8

Step 3: Identify controls and estimate control effectiveness Key controls: - Contractual SLAs for reporting timelines (contract in place) - Validated safety database with documented change control (validation report available) - Redundant business continuity arrangement (secondary site, but untested) - CAPA history: two minor findings closed in past year

Control effectiveness estimate: Medium (apply -10% reduction)

Step 4: Residual score calculation - Residual score = Initial score × (1 − control reduction) - Residual = 28.8 × 0.9 = 25.92

Step 5: Map to thresholds and determine classification - Residual score 25.92 → Critical (threshold ≥ 14) - Governance actions triggered per policy: - Immediate QPPV notification (done; date/time stamped) - Schedule on-site audit within 60 days (audit plan approved) - Implement monthly KPI reporting to QPPV and PV governance forum - Develop and test contingency/transition plan within 90 days - Ensure documented subcontractor mapping and subcontractor assessments

Step 6: Documentation and sign-off - Completed scoring spreadsheet with factor-level rationale (who, when, evidence) - Execution of governance triggers documented in meeting minutes (QPPV, QA, Head of PV) - Audit scheduled and audit scope defined with planned dates and resource allocation - Risk register entry created and assigned to vendor manager (see template below)

Inspection relevance and evidence

• Inspectors will expect to see the scoring sheet with narrative justification for each raw score and the controls considered.
• Demonstrable evidence of the controls (e.g., validation report, BCP test results, contractual SLA with timelines) must be linked.
• Evidence of escalation (emails/meeting minutes) and actions taken (audit plan, contingency plan) is necessary to show that the organisation treats the classification as operationally meaningful.

Notes on practical implementation

• Keep the scoring spreadsheet manageable and auditable (time-stamped file or VCS).
• Require narrative justification entries for any score >3 to demonstrate thought process.
• Set a governance review (e.g., quarterly) of weighting factors and thresholds to ensure they remain aligned with regulatory expectations and organisational risk appetite.
• Traceability: include links or reference IDs to the contract, audit report, KPI dashboard and CAPA tickets in the scoring spreadsheet.

C. Template risk register entry (illustrative operational model)

Below is a template risk register entry suitable for inclusion in a central vendor risk register (spreadsheet/SDR). After the blank template, a completed example using Vendor A (from the worked example) is provided.

Template fields (mandatory fields bolded; optional fields noted) - ID: (unique identifier) - Vendor name (legal entity): ** - Service(s) provided: ** - Product(s)/MAH units supported: ** - Contract start / end date: ** - Vendor manager (owner): ** - Assessment date: ** - Assessor(s): ** (name and role) - Initial scores (per factor) and narrative justification: ** (Regulatory Impact, Patient Safety Impact, Data Integrity Impact, Business Continuity Impact, Operational Complexity) - Weights applied (if any): (documented in governance) - Initial weighted score: ** - Controls in place (summary): ** (contractual SLAs, validation, redundancy, KPIs) - Control effectiveness estimate (High/Medium/Low) and method of determination: ** - Residual score: ** - Classification: ** (Low/Medium/High/Critical) - Risk treatment: ** (accept/mitigate/transfer/avoid) - Mitigation actions required (action, owner, target date, evidence link, status): ** - Audit requirement and next audit date: ** - KPIs assigned (list) and reporting frequency: ** - Contingency/transition plan status (None/Draft/Completed/Tested): ** - Subcontractors used (Y/N) and status of subcontractor oversight: ** - Historical issues (audit findings/CAPAs) and closure status: ** - Inspection evidence folder link/ID: ** (point to compiled documents) - Last reviewed: ** (date) - Next review due: ** (date) - Priority: ** (Low/Medium/High/Critical) - QPPV notified (Y/N) and date of notification: ** - QA sign-off (name, date): ** - Notes: (free text for context)

Sample filled entry — Vendor A (Global Case Processing Vendor)

• ID: VEND-2026-001
• Vendor name (legal entity): GlobalPV Services Ltd.
• Service(s) provided: Case intake, triage, ICSR entry, expedited reporting to regulators, follow-up case management.
• Product(s)/MAH units supported: Products A, B, C (EU centralised, multiple national procedures).
• Contract start / end date: 2022-06-01 / 2027-05-31
• Vendor manager (owner): J. Smith, Vendor Management Office
• Assessment date: 2026-05-10
• Assessor(s): J. Smith (Vendor Manager), L. Patel (Head of PV Risk), S. Nguyen (QA Lead)
• Initial scores and rationale:
• Regulatory Impact: 5 — performs expedited reporting; direct contact with regulators.
• Patient Safety Impact: 5 — performs triage and seriousness assessments affecting case follow-up.
• Data Integrity Impact: 4 — primary data entry into safety database; hosts database.
• Business Continuity Impact: 5 — sole provider for multiple products in EU.
• Operational Complexity: 4 — multi-country, multi-language processing.
• Weights applied: Regulatory Impact (1.5), Patient Safety (1.5), Data Integrity (1.2), Business Continuity (1.0), Complexity (1.0)
• Initial weighted score: 28.8
• Controls in place: Contractual SLA for 24-hour intake and expedited reporting, validated PV database (validation report VDB-VAL-2024-03), BCP with secondary site, monthly KPIs, documented SOPs for case handling.
• Control effectiveness: Medium — validation complete, SLA in contract, BCP untested to date.
• Residual score: 25.92
• Classification: Critical
• Risk treatment: Mitigate — enhance oversight and contingency testing.
• Mitigation actions:
• On-site audit scheduled: Owner S. Nguyen (QA); Target 2026-07-15; Evidence: Audit report AR-2026-07-15; Status: Planned.
• BCP test and failover drill: Owner J. Smith; Target 2026-08-30; Evidence: BCP-test-2026; Status: Planned.
• Monthly KPI dashboard to QPPV: Owner L. Patel; Target 2026-06-01 (start); Evidence: KPI report links; Status: Ongoing.
• Confirm subcontractor list and assessments: Owner J. Smith; Target 2026-06-15; Evidence: Subcontractor assessment folder; Status: In progress.
• Audit requirement and next audit date: On-site audit within 60 days (2026-07-15)
• KPIs assigned: ICSR timeliness (% within 24h), ICSR quality (error rate), follow-up completeness, CAPA closure time; reporting frequency: monthly
• Contingency/transition plan status: Draft — to be completed and tested by 2026-08-30
• Subcontractors used: Yes — translator services (assessments pending)
• Historical issues: Two minor findings (2024) closed; no open CAPAs
• Inspection evidence folder link/ID: PV-Vendor-GlobalPV-2026-05 (index.pdf)
• Last reviewed: 2026-05-10
• Next review due: 2026-11-10 (six-month cadence for critical vendors)
• Priority: Critical
• QPPV notified: Yes — 2026-05-12 (email and meeting minutes PV-GOV-2026-05)
• QA sign-off: S. Nguyen, QA Lead — 2026-05-12
• Notes: Executive steering committee alerted 2026-05-13; contingency planning to include transition to alternate provider shortlist.

Governance and operational guidance for the register

• Ownership: Each vendor entry must have a named owner (Vendor Manager) accountable for accuracy and updates.
• Sign-off: All initial risk assessments require QA and QPPV sign-off for critical/high classifications.
• Review cadence: Low = annual, Medium = annual or biannual, High = biannual, Critical = quarterly or 6-monthly with monthly KPI reporting.
• Version control: Maintain historic entries with timestamps and retain previous control evidence for inspection (minimum retention per records policy).
• Escalation matrix: Define thresholds that automatically trigger notifications to QPPV, Head of PV, and Head of QA (documented in PV governance SOP).
• Integration: Ensure the risk register is linked to audit planning tools, CAPA systems and KPI dashboards for traceability.

Inspection relevance

• Inspectors expect to see the register populated and kept current with completed actions, not just initial assessments.
• Demonstrate the linkage between a vendor's risk classification and concrete oversight actions (audit schedule, KPIs, contingency plans).
• Provide a printed or PDF extract of the vendor entry, with hyperlinks replaced by the document IDs so paper and electronic evidence are both available.

D. Implementation checklist and governance controls (practical steps)

To operationalise the appendix and ensure inspection readiness, follow these steps:

1. Adopt or adapt the scoring model and thresholds into your PV vendor oversight SOP.
2. Define roles and sign-off authorities (Vendor Manager, QA, QPPV, Head of PV) and incorporate them into the SOP.
3. Create a standardised scoring spreadsheet template with mandatory narrative fields and evidence links.
4. Establish a centralised vendor risk register (controlled document or GRC tool) with the template fields described.
5. Define review cadences by category and automate calendar reminders for reassessment.
6. Link the register to audit planning and CAPA management systems to ensure actions are traceable.
7. Prepare an inspection pack template (1-page executive summary, score sheet, contract, last audit, KPIs) and store it with each vendor file.
8. Conduct periodic internal quality reviews of the vendor oversight process and document outcomes.
9. Test contingency plans for critical vendors and retain test evidence and lessons learned.
10. Maintain version control and retention consistent with your global record retention policy.

Regulatory context and mapping

• EMA GVP Module I: quality systems and requirement for evidence of controls and governance.
• EMA GVP Module III: inspectors will examine the vendor oversight framework and audit evidence.
• ICH Q9: foundational QRM concepts underpinning scoring, residual risk, and control effectiveness.
• Regional pharmacovigilance regulations: ensure reporting obligations and delegated responsibilities are reflected in contracts and assessments.

Closing note on inspection preparedness

• Inspectors seek evidence that decisions are implemented, not only theorised. Use the appendix artefacts to provide a compact, auditable trail linking risk identification to mitigation, monitoring, governance and measurable outcomes. Keep an example inspection pack ready for each critical vendor.

References

1. EMA Good Pharmacovigilance Practices (GVP) Module I – Pharmacovigilance Systems and Their Quality Systems.

Regulatory Note

This article is educational. Binding obligations arise from applicable legislation and marketing-authorisation conditions. GVP and national-authority publications describe regulatory expectations. Suggested models, frequencies, thresholds, scorecards, matrices, checklists and scenarios are illustrative or recommended practice unless a legal provision is expressly identified.

Commission Implementing Regulation (EU) No 520/2012 was amended; use the consolidated text applicable from 12 February 2026 with current guidance. EMA states that affected GVP modules will be revised. Verify current legislation, guidance, national requirements, contracts and product-specific commitments before operational use.