Disproportionality Analysis in Pharmacovigilance

Introduction

Modern pharmacovigilance systems generate very large volumes of safety data. National spontaneous reporting databases, EudraVigilance, the FDA Adverse Event Reporting System (FAERS) and other safety databases may contain millions of Individual Case Safety Reports (ICSRs). Manual review of every possible product-event combination is therefore impractical.

Disproportionality analysis was developed to assist with signal detection in large safety databases. The objective is to identify product-event combinations that are reported more frequently than would be expected based on overall reporting patterns within the database.

These methods are widely used by regulatory authorities, Marketing Authorisation Holders and signal management teams. They are particularly useful as screening tools because they can rapidly identify observations that may warrant further evaluation.

However, disproportionality analyses do not establish causality. A statistical association identified through disproportionality analysis should be regarded as a hypothesis requiring further investigation rather than evidence that a medicinal product causes a particular event.

Purpose of Disproportionality Analysis

Disproportionality analysis seeks to identify reporting patterns that differ from what would be expected if no association existed between a medicinal product and an event.

The basic principle is straightforward.

If a particular adverse event is reported more frequently for a product than would be expected relative to other products in the database, the observation may warrant further review.

The output of disproportionality analysis is therefore a potential signal rather than a regulatory conclusion.

These methods are intended to support signal detection activities and should be interpreted within the broader context of signal validation and signal assessment.

Disproportionality Analysis Within Signal Management

Disproportionality analysis forms part of signal detection rather than signal assessment.

A simplified relationship is:

Safety Database
        ↓
Disproportionality Analysis
        ↓
Potential Statistical Signal
        ↓
Signal Validation
        ↓
Signal Assessment

The statistical output generated by disproportionality methods is only one source of information.

Clinical review, literature findings, epidemiological evidence and biological plausibility remain important components of signal management.

A product-event combination may generate a disproportionality signal but ultimately be rejected following scientific review.

Conversely, some important safety concerns may be identified without producing a statistically significant disproportionality signal.

Reporting Frequencies and Expected Counts

Most disproportionality methods compare observed reporting frequencies with expected reporting frequencies.

Observed reporting frequency refers to the number of reports involving a particular product-event combination.

Expected reporting frequency represents the number of reports that would be anticipated if the product and event occurred independently within the database.

When observed counts substantially exceed expected counts, a disproportionality signal may be generated.

The precise method used to perform this comparison depends upon the statistical approach employed.

Reporting Odds Ratio (ROR)

The Reporting Odds Ratio (ROR) is one of the most widely used disproportionality measures.

The ROR compares:

• Reporting of a specific event for a product
• Reporting of the same event for all other products

The measure is conceptually similar to an odds ratio used in epidemiological studies.

An elevated ROR suggests that the event is reported relatively more frequently for the product of interest than for comparator products within the database.

Interpretation requires caution.

An elevated ROR does not demonstrate that the product causes the event. It indicates only that reporting patterns differ from expectation.

Confidence intervals are typically used when evaluating statistical significance.

Proportional Reporting Ratio (PRR)

The Proportional Reporting Ratio (PRR) is another commonly used disproportionality measure.

PRR compares:

• The proportion of reports involving a specific event for a product
• The proportion of reports involving the same event for all other products

The methodology is relatively simple and has historically been used in several pharmacovigilance systems.

As with ROR, an elevated PRR indicates disproportionate reporting rather than causality.

Interpretation should always consider data quality, reporting biases and clinical plausibility.

Information Component (IC)

The Information Component (IC) is a Bayesian disproportionality measure developed for large pharmacovigilance databases.

The method is widely associated with the signal detection framework developed by the Uppsala Monitoring Centre.

The IC evaluates whether observed reporting exceeds expected reporting while incorporating statistical shrinkage techniques designed to reduce instability when report counts are small.

This approach can improve performance when evaluating rare events or product-event combinations supported by limited numbers of reports.

Interpretation remains conceptually similar to other disproportionality methods.

A positive signal indicates disproportionate reporting, not proof of causation.

Empirical Bayesian Methods

Several pharmacovigilance systems utilise empirical Bayesian methodologies.

Examples include:

• Empirical Bayesian Geometric Mean (EBGM)
• Multi-item Gamma Poisson Shrinker (MGPS)

These methods attempt to improve signal detection performance by reducing the influence of random variation, particularly when dealing with sparse data.

Bayesian approaches are often considered more stable than simple disproportionality measures when evaluating large databases containing many rare product-event combinations.

The choice of methodology varies between organisations and regulatory systems.

Strengths of Disproportionality Analysis

Disproportionality methods offer several advantages.

They can evaluate very large datasets efficiently and identify observations that may not be apparent through routine case review.

Additional advantages include:

• Scalability
• Consistent application
• Automated screening capability
• Detection of unusual reporting patterns
• Support for routine surveillance activities

These characteristics make disproportionality analysis a valuable component of signal detection programmes.

However, the strengths of the methodology should be understood alongside its limitations.

Limitations of Disproportionality Analysis

Disproportionality analysis has important limitations that must be considered during interpretation.

The most important limitation is that spontaneous reporting systems do not provide true incidence rates.

The number of reports received does not represent the number of patients exposed to the product and does not necessarily reflect the actual frequency of an adverse reaction.

As a result, disproportionality measures should not be interpreted as measures of risk.

Additional limitations include:

• Under-reporting
• Reporting bias
• Stimulated reporting
• Missing information
• Duplicate reports
• Confounding factors
• Data quality variability

These limitations mean that statistical outputs require clinical and scientific review before conclusions are reached.

False Positives

False positives are common within disproportionality analysis.

A product-event combination may generate a statistical signal despite the absence of a genuine causal relationship.

Several factors may contribute to false positives.

Examples include:

• Media attention
• Regulatory warnings
• Class effects
• Reporting trends
• Public awareness

Statistical significance therefore does not necessarily imply clinical significance.

Many disproportionality signals are ultimately closed following validation and assessment.

False Negatives

False negatives are equally important.

Some genuine safety concerns may fail to generate a disproportionality signal.

Possible reasons include:

• Rare events
• Limited reporting
• Delayed recognition
• Small exposure populations
• Methodological limitations

For this reason, effective signal detection programmes do not rely solely upon disproportionality analysis.

Medical review and other signal detection approaches remain necessary.

Clinical Review of Statistical Signals

Statistical outputs should be regarded as the beginning of the review process rather than the end.

Once a potential signal is identified, reviewers may evaluate:

• Case narratives
• Clinical plausibility
• Biological mechanisms
• Literature evidence
• Epidemiological findings
• Alternative explanations

This review determines whether the observation should proceed to signal validation and assessment.

Clinical judgement therefore remains central to signal management despite increasing use of statistical methods.

Disproportionality Analysis in EudraVigilance

Disproportionality methodologies are used within EudraVigilance signal detection activities.

The EudraVigilance Data Analysis System (EVDAS) supports identification of product-event combinations that may warrant further evaluation.

These outputs contribute to regulatory signal management activities performed by Member States, EMA and Marketing Authorisation Holders.

However, EVDAS outputs should not be interpreted as confirmed regulatory signals.

Further review remains necessary before conclusions are reached.

Disproportionality Analysis During Inspections

Inspectors may review how disproportionality analyses are incorporated into signal detection programmes.

Areas commonly examined include:

• Methodology selection
• Threshold definitions
• Review processes
• Validation activities
• Documentation practices
• Governance arrangements

Inspection findings are often related to deficiencies in review processes or documentation rather than the specific statistical methodology employed.

Organisations should therefore be able to explain both how statistical outputs are generated and how they are evaluated.

Common Misunderstandings

Several misconceptions frequently arise.

One misconception is that disproportionality analysis measures the incidence of adverse reactions. It does not.

Another is that disproportionality signals establish causality. They do not.

A third misconception is that signal detection can be fully automated through statistical methods. In practice, medical review and scientific judgement remain essential.

Finally, some organisations overemphasise disproportionality outputs while underutilising qualitative review methods. Effective signal detection programmes generally combine multiple complementary approaches.

Key Takeaways

Disproportionality analysis is a statistical screening technique used to identify product-event combinations reported more frequently than expected within pharmacovigilance databases.

Common methodologies include ROR, PRR, Information Component and empirical Bayesian approaches.

These methods support signal detection but do not establish causality or measure incidence.

Statistical outputs require clinical review, validation and assessment before regulatory conclusions can be reached.

Effective signal detection programmes combine disproportionality analysis with qualitative review and broader scientific evaluation.

References

1. EMA Good Pharmacovigilance Practices (GVP) Module IX – Signal Management.
2. EudraVigilance Data Analysis System (EVDAS) Guidance.
3. CIOMS VIII Practical Aspects of Signal Detection in Pharmacovigilance.
4. Bate A, Evans SJW. Quantitative Signal Detection Using Spontaneous ADR Reporting.
5. Hauben M, Aronson JK. Defining Signal and Its Subtypes in Pharmacovigilance.
6. van Puijenbroek EP et al. A Comparison of Measures of Disproportionality for Signal Detection.
7. Uppsala Monitoring Centre. Bayesian Signal Detection Methodology.
8. ICH E2E Pharmacovigilance Planning.