A proportional reporting ratio, or PRR, is a statistical calculation intended to determine whether a particular adverse event, following vaccination, occurs frequently enough to constitute a safety concern, or “signal.” A PRR compares the relative frequency of reports of a particular adverse event following a particular vaccine versus the relative frequency of reports of the same adverse event following receipt of other vaccines. The PRR is the CDC's primary method of monitoring the VAERS database for “safety signals.”¹⁾

In the table below, A is the total number of a particular adverse event reported for the vaccine of interest; B is the total number of all other adverse events reported for the vaccine of interest; C is the total number of the same adverse event reported for a comparator vaccine, or vaccines; and D is the total number of all other adverse events reported for the comparator vaccine, or vaccines.²⁾

While a PRR greater than 1 indicates that a particular adverse event (e.g. fever, stroke, myocarditis, etc.) is reported with greater relative frequency when compared to other vaccines, the CDC defines a “safety signal” as a PRR greater than 2, with a chi-squared statistic of at least 4, and with 3 or more cases of the adverse event following receipt of the specific vaccine of interest.³⁾

Statisticians contend that the particular safety profile of the COVID-19 vaccines highlights significant drawbacks in using the PRR in pharmacovigilance.⁴⁾

For example, consider what would happen if a relatively dangerous vaccine were introduced that resulted in 20 times as many AEs of all types as all the other vaccines to which it gets compared. As can be seen here, even a very large number of adverse events may not result in a “safety signal,” because the PRR is invariant with respect to scale.⁵⁾

In the simplified example above, all of the hypothetical adverse events are increased twenty-fold, and thus the PRR calculations for a relatively safe vaccine and for a relatively dangerous vaccine yield the same result (because the numerator remains unchanged). In a more realistic example where the increase in adverse events is more uneven, impacting certain categories of adverse event, but not others, the PRR calculation will be slightly higher for certain adverse events associated with a more dangerous vaccine. Nonetheless, even where the increase in certain adverse events is 1000-fold (see below), the PRR changes very little.

As this slightly more realistic example shows, the numerator of the PRR changes only slightly when a handful of AEs scale up, and others do not, even for extreme values of the scalar, k. In this example, 4 different categories of adverse event are reported at frequencies of (55, 7, 6, 11) for one vaccine. For a second vaccine, the first two categories of adverse event are increased 20-fold, while the second two categories remain the same. For a third vaccine, the first two categories of adverse event are increased 1,000-fold, while the second two categories remain the same. As shown here, even with a 1,000-fold increase in certain adverse events, the numerator of the PRR only changes a little. Because the PRR's denominator (calculated from comparator vaccines) remains unchanged, the potential for yielding a “safety signal” also changes very little.⁶⁾

Because the PRR is scale invariant, important safety signals have come and gone without notice over the course of the vaccine rollout. For example, the MedDRA term “death” showed up as a signal in dispersion analysis in February 2021, but no longer does due to the rising quantity of so many other AEs. Rising numbers of adverse events other than “death” (as in this example), inflates the denominator, thereby forcing structural mean-reversion of the PPR function toward 1.⁷⁾

Despite the PRR's inherent shortcomings, including potential signal masking, PRR calculations for the COVID-19 vaccines using real-world data nonetheless result in clear safety signals. In a public comment submitted to the August 30, 2021 ACIP meeting on Emergency Use Authorization for the Pfizer vaccine for ages 16 and older, Josh Guetzkow calculates PRRs for such serious adverse events as death, Guillain-Barré syndrome, coagulopathy, myocardial infarction, and myopericarditis⁸⁾.

The PRRs in bold exceed the CDC's threshold defining a “safety signal.” These figures have been calculated according to the CDC's own procedures, using real-world data, yet the CDC insists that the adverse events reported through VAERS and vSafe do not represent a safety signal⁹⁾¹⁰⁾.

While the FDA and CDC now acknowledge that myocarditis is causally linked to the COVID-19 vaccines, a safety signal for the condition was not discovered through routine monitoring of the VAERS and v-safe databases using PRR calculations. Indeed, even as vaccine recipients - especially teenage boys - continue to be hospitalized with vaccine-induced myocarditis, the condition has still not elicited a safety signal as defined by the CDC¹¹⁾.

The use of PRR (and similarly ARR) is most appropriate when examining adverse events associated with a heterogeneous pool of drugs or other therapies. For instance, a paper by Stephen Evans, et al. uses PRR to detect excess proportions of AEs among 15 newly-marketed drugs of different kinds and purposes while detecting 481 safety signals¹²⁾. However, for certain drugs or therapies, the PRR fails altogether to identify adverse events, even when the total magnitude or number of adverse events is quite large. Most notably, despite historically high magnitudes of VAERS reports associated with the COVID-19 vaccines, the CDC's PRR analysis has detected zero safety signals.

Part of the failure of PRR analysis to detect safety signals in the COVID-19 vaccines is due to a statistical phenomenon called “signal masking” or simply “masking.” In a 2017 paper, Hauben and Maigen explain how masking of high AE magnitude takes place in PRR analysis when drugs or therapeutics are primarily compared with similar medical interventions¹³⁾. In the case of the COVID-19 vaccines, this statistical masking means that PRR analysis (or what seems to be a highly similar ARR analysis used in the VSD RCA) would be unlikely to detect large numbers of adverse events as signals for concern. This view is confirmed by Wisniewski, et al. (including authors notably employed by Pfizer and Astra-Zeneca) who conclude that such signals “are not easily interpretable in terms of clinical impact,” and that “calculation of PRRs…should not replace nor delay the performance of epidemiological studies.”¹⁴⁾

An Adjusted Rate Ratio (ARR) is a variant of the PRR used by analysts working with the Vaccine Safety Datalink (VSD). The VSD is a collaborative project between the CDC and nine integrated health care organizations, most of which are part of the Kaiser Parmanente network.¹⁵⁾. The stated goal of the VSD is to use Rapid Cycle Analysis (RCA) to “detect adverse events following vaccination in near real time so the public can be informed quickly of possible risks.¹⁶⁾” The VSD RCA does not completely define what it calls Adjusted Rate Ratios, but the resulting statistics appear to be PRRs with some form of adjustments for data collection sites, demographics, etc. These adjustments, however they are calculated, have nothing to do with the problems inherent in the use of PRR in safety signal detection, however.