Signal Management Metrics

A signal-management system produces many measurements: observations screened, validation decisions, assessments opened, actions completed, ageing of open work and quality-review results. These measurements become useful only when their definitions are stable, their source data are reliable and their interpretation answers a meaningful operational or governance question.

Purpose and Regulatory Context

EU pharmacovigilance legislation and GVP require marketing-authorisation holders to operate an effective pharmacovigilance system and quality system. GVP Module IX — Signal management describes the signal process, while GVP Module I — Pharmacovigilance systems and their quality systems provides the wider framework for performance monitoring, quality control and management review.

Neither EU legislation nor GVP prescribes a universal signal-metrics catalogue, fixed percentages, traffic-light thresholds or dashboard design. These are organisational controls. Their value is determined by whether they help the MAH detect loss of control, understand capacity and quality, support oversight and take action when performance changes.

As of September 2026, GVP Module IX Rev. 1 remains published, but the legal framework has been amended by Commission Implementing Regulation (EU) 2025/1466. EMA has stated that Module IX will be revised for alignment. Metric definitions should therefore be reviewed when underlying regulatory processes change.

Metric, KPI and Quality Indicator

A metric is any defined measurement. A key performance indicator (KPI) is a deliberately selected metric used for management attention or decision-making. A quality indicator focuses specifically on whether work meets a defined quality expectation.

For example:

The distinction matters because collecting many metrics does not create effective oversight. A dashboard containing fifty measurements can still fail to show the one deteriorating process that requires intervention.

Start With the Management Question

A useful metric begins with a question rather than an available database field.

Examples include:

• Is the volume of incoming signal work changing faster than capacity?
• Are high-priority assessments ageing differently from routine work?
• Are closure rationales consistently documented?
• Are outsourced activities creating recurrent delays or quality defects?
• Do downstream actions arising from signals reach implementation?

Once the question is clear, the numerator, denominator, time point, population and exclusions can be defined. This sequence prevents the common error of measuring what is easy rather than what is important.

Building a Metric Definition

A reproducible metric should normally define:

• the process or risk question;
• numerator and denominator where applicable;
• inclusion and exclusion criteria;
• the relevant date field or time interval;
• status definitions;
• source system or record;
• calculation rules;
• treatment of missing or corrected data;
• reporting level, such as global, product, affiliate or vendor; and
• how the result will be interpreted.

A metric such as “assessment timeliness” is too vague. A reproducible definition might specify the population of assessments closed during the reporting period, the start and end timestamps, how paused work is treated, and which internal target is being compared with actual performance.

Denominators Matter

Many misleading metrics arise from poorly chosen denominators. A high completion percentage may look favourable simply because difficult open items have been excluded from the calculation. Conversely, including administrative duplicates or transfers can make performance appear worse than it is.

For a proportion, the denominator should represent the population about which the organisation is making a claim. If the question is whether high-priority assessments are completed within an internal target, the denominator should consist of the relevant high-priority assessments, not all signal records.

The denominator should also remain sufficiently stable over time to permit meaningful trend interpretation. When definitions change, historical comparability should be assessed rather than assumed.

Snapshot Metrics and Event Metrics

Two basic metric designs are often confused.

A snapshot metric describes the state of the system at a specified time. Examples include open assessments, overdue actions or the age distribution of the current backlog.

An event metric counts or analyses events occurring during a period. Examples include assessments closed during the month, signals escalated during a quarter or actions completed during the reporting interval.

Both can be useful, but they answer different questions. A system can close many assessments during a month while the backlog still grows because even more work is entering the process. Using both flow and stock measures often gives a more accurate view of system performance.

Data Provenance and Reproducibility

A metric is only as trustworthy as the data and transformation logic used to produce it. The organisation should be able to explain where the source data originated, how records were selected, what transformations were applied and whether later corrections can change the result.

This does not mean that every signal metric requires a separately validated analytics platform. It means that the controls should be proportionate to the importance of the metric and the risk of error. A metric used only for local workload planning may need simpler controls than one used to support senior governance decisions about potential loss of pharmacovigilance-system control.

Useful controls may include:

• defined source-system fields and status mappings;
• controlled calculation logic;
• documented handling of duplicates and missing dates;
• versioning when definitions change;
• reconciliation or sample checking of automated outputs;
• retention of sufficient source evidence to reproduce material results; and
• investigation of material discrepancies.

Common Metric Families

Signal-management metrics can be grouped conceptually rather than treated as a catalogue of mandatory indicators.

Workload and inventory

These describe the amount and composition of work, such as incoming observations, open validated signals, active assessments or outstanding downstream actions. They help identify changes in demand and capacity but do not by themselves show quality.

Timeliness and ageing

These examine elapsed time and backlog structure. Median or percentile measures can be more informative than averages when a few very old records distort the distribution. Age bands can reveal whether a backlog is newly created or chronically unresolved.

Internal timelines should be distinguished from externally binding timelines. A company may choose targets to control its process, but those targets should not be described as EMA deadlines unless an authoritative source establishes them.

Quality and completeness

These assess whether records contain the evidence, reasoning and documentation expected by the organisation. Examples include sampled completeness of assessment rationale, linkage between decisions and evidence, or quality-review outcomes.

Quality measures should avoid creating a superficial “box-completion” culture. The important question is whether the record supports scientifically defensible reconstruction of the decision.

Escalation and governance

These may examine unresolved escalations, ageing of important actions, or whether issues requiring a defined governance route were handled as intended. Such metrics can reveal whether the formal governance design operates in practice.

Downstream implementation

Signal management does not end when an assessment is closed. Measures may therefore follow actions into product information, RMP updates, additional pharmacovigilance, risk minimisation or regulatory responses where relevant.

A closed signal with an overdue safety action remains a live governance concern even if the signal-management tracker itself shows “complete.”

Metric Interpretation Requires Context

No single direction is always good or bad. An increase in validated signals might indicate improved surveillance, a genuine change in the safety environment, a coding change, a new product launch or excessive sensitivity in screening. A low number of signals could reflect a stable portfolio or an ineffective detection system.

Interpretation should therefore consider:

• product launches and indication changes;
• exposure growth or contraction;
• regulatory or procedural changes;
• database migrations or coding changes;
• changes in screening methodology;
• vendor or organisational transitions;
• major public safety communications that stimulate reporting; and
• changes in staffing or workload.

A metric should prompt a question before it prompts a conclusion.

Trend Analysis Versus Point Estimates

Single-period values can be misleading. Trend analysis can show whether deterioration is persistent, transient or associated with a known event.

Useful approaches include:

• monthly or quarterly time series;
• ageing distributions;
• rolling medians;
• stratification by product, region, vendor or priority; and
• comparison before and after a material process change.

Statistical process-control methods can sometimes help, but they are not required merely because the process is regulated. The sophistication of the method should match the decision need.

Targets, Thresholds and Tolerances

An internal target is a management tool. It should be justified by the process, risk, capacity and regulatory context.

Arbitrary universal targets such as “95% within seven days” or “zero assessments older than 30 days” are not general EU requirements. They may be sensible controls for a particular organisation, but the rationale should be explicit.

When thresholds are used, the governance response should also be defined. A red dashboard indicator that triggers no analysis or action is largely decorative.

Possible responses include:

• investigation of data quality;
• workload or resource review;
• focused quality sampling;
• process redesign;
• vendor escalation;
• CAPA where a quality-system deficiency is identified; or
• temporary enhanced oversight.

Metrics for Outsourced Signal Activities

Outsourcing can complicate measurement because the MAH and vendor may use different systems and definitions. A useful oversight model aligns key definitions at the interface and avoids relying solely on vendor-reported percentages.

The MAH should understand what the vendor metric actually measures, which records are excluded, and how exceptions are handled. Where a vendor metric is important to system oversight, the MAH should be able to reconcile it with its own records sufficiently to trust the conclusion.

Volume alone should not dictate oversight intensity. A low-volume outsourced activity can still be high risk if it involves important signal decisions or safety-critical information flows.

Relationship With Quality Management

Metrics can reveal deterioration but do not automatically establish a quality-system deficiency. A missed internal target may be an isolated workload event, whereas a persistent pattern may indicate inadequate resources, unclear responsibilities or ineffective process design.

The appropriate sequence is:

measurement → interpretation → investigation where warranted → action proportionate to the cause → follow-up measurement.

This prevents every adverse metric from becoming an automatic CAPA while still ensuring that meaningful deterioration receives structured attention.

Potential Failure Modes

The following are illustrative failure modes rather than reported inspection findings.

Inspection Considerations

An inspector may examine whether metrics support rather than obscure understanding of the signal-management system. Useful questions include:

• How is this metric defined and can it be reproduced?
• Which source systems and statuses feed the calculation?
• Have definitions changed, and how was comparability managed?
• Why was this target chosen?
• What happens when the threshold is breached?
• Can an adverse trend be traced to investigation and management action?
• How are vendor-reported results verified?
• Are important signal actions visible after the assessment itself is closed?

The strongest evidence is not an elaborate dashboard. It is a consistent chain from reliable measurement to interpretation and proportionate action.

Practical Metric Design Checklist

The following is recommended operational practice rather than an EMA-required template.

1. What management or risk question is the metric intended to answer?
2. Is the numerator defined precisely?
3. Does the denominator represent the population about which the conclusion is being made?
4. Is the measure a snapshot or an event metric?
5. Are source fields, statuses and exclusions defined?
6. Can material results be reproduced from source data?
7. Are missing data and corrections handled consistently?
8. Can historical values still be compared after a definition change?
9. Is stratification available when aggregate results conceal important variation?
10. Is an internal target clearly distinguished from a regulatory deadline?
11. Is a response defined when the result indicates deterioration?
12. Does the metric remain useful enough to justify the reporting burden?

Key Takeaways

Signal-management metrics are measurement tools within the pharmacovigilance quality system, not a prescribed regulatory scorecard.

Reliable metrics require stable definitions, appropriate denominators, trustworthy source data and enough provenance to understand how the result was produced.

Counts, percentages and averages should be interpreted in context. Changes in portfolio, methods, systems or reporting behaviour can alter a metric without representing a true change in safety-system performance.

KPIs are the smaller subset of metrics selected for management attention. The companion article [[signal-management-kpis]] addresses that selection and dashboard perspective in greater depth.

The regulatory value of a metric lies in what the organisation learns and does with it, not in whether it resembles a particular industry benchmark.

References

1. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module I — Pharmacovigilance systems and their quality systems. EMA/541760/2011.
2. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module IX — Signal management (Rev. 1). EMA/827661/2011 Rev. 1.
3. European Medicines Agency. Questions and answers on signal management. EMA/261758/2013 Rev. 5, updated January 2026.
4. European Medicines Agency. Good pharmacovigilance practices (GVP). Current module-status page.
5. European Union. Commission Implementing Regulation (EU) No 520/2012, as amended by Commission Implementing Regulation (EU) 2025/1466.
6. European Union. Directive 2001/83/EC, as amended.
7. European Union. Regulation (EC) No 726/2004, as amended.

Regulatory Note

EU legislation and GVP require an effective pharmacovigilance quality system and controlled signal management, but they do not prescribe universal signal-management metrics, percentages, thresholds or dashboard designs. The metrics discussed here are examples of operational controls. As of 8 September 2026, GVP Module IX Rev. 1 remains published while EMA prepares revisions following Commission Implementing Regulation (EU) 2025/1466.