Who Cares About VAERS

June 22, 2022 6:58 PM

If anybody and everybody can report any kind of supposed vaccine side effects to VAERS, what good is it?

I'm having trouble understanding how VAERS (Vaccine Adverse Event Reporting System) can be useful if anybody and everybody can report whatever they want to it - real, fake, accurate, imagined, etc.

How do the scientists, etc. who use the reported data make use of it when there's no way of knowing for sure whether any of it has any integrity at all? Isn't this a recipe for garbage in garbage out?

posted by Dansaman to Health & Fitness (8 answers total)

Interesting question! I enjoyed reading this ~article from Johns Hopkins Bloomberg School of Public Health:
What VAERS Is (And Isn't)

Relevant bits:

"VAERS serves as an early warning system for unforeseen problems with approved vaccinations that might be worth investigating scientifically. Often, these problems are so rare that they don’t appear until after clinical trials when a much larger population receives vaccinations.

VAERS is great at identifying signals of potential concern, says Kawsar Talaat, MD, an associate professor in International Health and co-director of clinical research for the Institute for Vaccine Safety. “Some of those signals end up panning out as true safety issues, and some don’t.”

For example, VAERS data helped doctors adjust the childhood polio vaccine schedule in 1997 in response to the 8–10 cases of vaccine-induced paralysis they learned had been occurring annually, according to the CDC’s published reports. That change greatly reduced the rare instances of severe side effects after polio vaccinations.

VAERS data also first surfaced reports of myocarditis following the second dose of COVID-19 mRNA vaccine. After extensive review, scientists determined that the benefits of the vaccine ultimately outweigh the increased risk of myocarditis observed in some of the vaccinated (primarily males ages 12–29).

Moreover, additional studies show the risk of myocarditis to be 16 times greater among those infected with COVID-19 than the uninfected, suggesting that full vaccination is helpful in preventing myocarditis and other complications of the disease.

...Talaat says the best source of research stemming from VAERS is the CDC, because they are able to trace the records backward and verify them.

For example, by January 10, 2021, VAERS logged 1,266 reports of adverse events following the Moderna vaccine. The CDC and FDA flagged 108 of those cases for further review. Ultimately, 10 of those cases turned out to be anaphylaxis, a severe allergic reaction, with nine of the affected people having a history of allergic reactions or allergies—including five of those nine with a history of anaphylaxis specifically. This screening allowed doctors to advise vaccination sites to continue following CDC guidance for administering vaccines as they had been.

...When researchers notice a pattern, such as an uptick in side effects after a particular vaccination or among a particular group of patients, such as women over 65 or people with diabetes, they can follow up by investigating with other safety monitoring systems, such as the Vaccine Safety Datalink, which can connect adverse events to medical records and reports from health care facilities and practitioners. Importantly, in addition to being verified, this data includes controls, Talaat says. That's because medical data for women over 65 or diabetics would include reports from both vaccinated and unvaccinated patients.

VAERS doesn’t reveal how many people report the same reaction, nor how many in an unvaccinated population report the same thing. By following up with other monitoring systems, researchers can determine if, for example, the population of people who report getting arthritis after a vaccine is the same as a control group getting arthritis in the same period, which would rule out the vaccine as the cause."
posted by rrrrrrrrrt at 7:04 PM on June 22, 2022 [9 favorites]

I am finding this presentation FASCINATING

The main data-use point I understand it to be making is that VAERS is valuable for identifying changes and anomalies - if a vaccine that's been delivered for 20 years with no reported deaths gets reports of 3 deaths in the first quarter of the year, something is worth looking at. Similarly, if report numbers are changing *and* clustering around vaccination lot numbers, particular demographic groups, particular symptoms, etc., those are signals that can be investigated. Here's one example they give:

Reporting rates example: RotaShield® and intussusception (results - by May 1999)

9 cases of intussusception reported to VAERS
 8/9 cases after dose 1
 8/9 cases within 1 week of vaccination
 Median age 4 months
 5 required surgical intervention
By comparison
From Nov 1990 - Nov 1998 in VAERS
 Only 3 cases intussusception reported following receipt
of any other vaccine

Reporting rates example: RotaShield® and intussusception (observed v. expected cases)
Through July 1999
 Assumptions
 1.5 million doses of RotaShield® administered
 Background rate: 51/100,000 infant-years*
 Expected: 14-16 cases within 1 week of vaccination by
chance alone
 Observed: 12/15 VAERS reports with onset <1> after vaccination
 Know VAERS reporting sensitivity <>  Reporting to VAERS that approaches background rate is
concerning due to known underreporting to VAERS
posted by rrrrrrrrrt at 7:38 PM on June 22, 2022 [1 favorite]

VAERS is NOT the only vaccine safety monitoring system in use. There are like 3 others.

VAERS is useful to spot common signals that came from multiple parts of the country, instead of only from new vaccines, and so on. It's useful to spot the "delta", if you will. Basically, something had changed. And this will suggest other vaiable to filter and study.

By itself, it doesn't mean ANYTHING. Even the signals spotted within VAERS don't mean anything, merely "that's interesting, let's look into it further".

If there are enough commonalities to warrant a study, the first thing the folks will do is go to the reporting source and seek evidence. And that's when the bogosities will drop out.

The folks who rely on VAERS are well versed with antivax tactics such as stuffing VAERS with bogus entries. Last year, they found that a majority of reporting supposedly heart inflammation in kids after COVID vaccine came from only a handful of computers, and vast majority of those reports cannot be substantiated by hospital records in the area.
posted by kschang at 7:49 PM on June 22, 2022 [5 favorites]

Imagine you make potato chips, and as a big international potato chip concern, you get periodic complaint calls from people complaining that they fell ill after eating your chips, that their chips were defective or mislabeled or contaminated, or that something else is wrong with your products. Anyone can make those calls, and you largely aren't verifying the claims—some people maybe got sick because they ate bad clams after having chips, some people like to complain if they think the chips should be a different size or shape, and some people just like to make up stories to try to score some free chips—but you still take all the reports so you can look for patterns. You can't look at the phone calls and conclude that a certain percentage of your customers will get stomachaches after eating your chips, but if you get an abnormal number of reports of people falling ill after eating batch #1234, you'd want to investigate that further by other means (you might obtain and test a sample of product from that batch or review medical records from those who are sick, for example). You might also have different reactions based on the nature and credibility of the reports; if the head of a school lunch program calls in with video evidence of sealed bags of chips containing metal pieces causing serious injury, you'd likely react differently than you would to a parent calling in to complain their kid ate four giant bags of chips and now has a sore tummy.

VAERS is basically that complaint hotline. An individual report doesn't tell you that there's a problem, but the entire stream of reports provides a signal that something may be worthy of more study: "VAERS cannot determine if a vaccine caused an adverse event, but can determine if further investigation is needed". Anyone can submit a report to VAERS, but many reports are submitted by health care providers or vaccine manufacturers, and providers are required by law to submit certain types of adverse events and encouraged to report others. Reports from health care providers are still just case reports, but most aren't likely to be totally fictitious, and they may include meaningful clinical data (and analysts will follow up on serious reports by obtaining medical records, autopsy records, etc...).

It might be interesting to look at some of the VAERS publications to see how the system is used. For example, Shingrix was licensed for use in the US, and the CDC reviewed VAERS data on its first eight months of use:

After licensure, CDC and FDA began safety monitoring of RZV in the Vaccine Adverse Event Reporting System (VAERS) (3). During the first 8 months of use, when approximately 3.2 million RZV doses were distributed (GlaxoSmithKline, personal communication, 2018), VAERS received a total of 4,381 reports of adverse events, 130 (3.0%) of which were classified as serious. Commonly reported signs and symptoms included pyrexia (fever) (1,034; 23.6%), injection site pain (985; 22.5%), and injection site erythema (880; 20.1%). No unexpected patterns were detected in reports of adverse events or serious adverse events. Findings from early monitoring of RZV are consistent with the safety profile observed in prelicensure clinical trials.

In short, they looked for patterns that would have been unexpected based on knowledge of what's typical in vaccines in general and based on the vaccine's clinical trials, and they didn't find anything unusual. For example, one thing they did was to compare the volume and types of reports for this vaccine to the reports for other vaccines ("Empirical Bayesian data mining methods were used to identify RZV-adverse event pairings that were reported at least twice as frequently as were reported in all other U.S.-licensed vaccines in the VAERS database"). If a particular adverse event is more common than expected, that's something they'd want to look into more. If they investigate and discover the reports are all garbage submitted by antivaxers trying to throw a wrench in the machine, then at least the question is answered.

Researchers might also look at the distribution of reports in particular populations. Are COVID vaccines safe for pregnant people? They weren't included in the clinical trials, so we weren't not too sure when the vaccines first came out, but we knew a number of people choose to get vaccinated during pregnancy, so researchers looked at VAERS data, along with the surveillance systems setup specifically for COVID vaccines and a pregnancy registry intended to help answer this question to quickly get preliminary data to see if there were significant safety concerns, providing initial information to help people make a difficult decision.

Maybe also take a look at the paper Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS). especially the section "How do CDC and FDA analyze VAERS data?":

The basic analyses of VAERS data are intended to detect concerning patterns or unusual and unexpected changes in adverse event reporting that might indicate a safety problem in a specific vaccine or vaccine type[...]VAERS is not able to provide incidence of adverse events. As a passive, numerator-only surveillance system, VAERS lacks information on total number of individuals vaccinated and total number who experience an adverse event, as well as incidence of adverse events in unvaccinated individuals. However, the proportion of reports involving a specific adverse event and a specific vaccine can be compared to the proportion of reports involving the same adverse event and other vaccines.

The "Closing thoughts" section of the paper also describes some instances where potentially concerning patterns were found in VAERS data, which prompted further investigation using other surveillance and epidemiological tools.
posted by zachlipton at 8:19 PM on June 22, 2022 [10 favorites]

I participated in the pregnancy specific registry zachlipton mentions above, and I participated in a couple of calls to answer structured interview questions, and also gave permission for the registry to contact my ob/gyn and delivery hospital. Not VAERS, but illustrates the kind of extra data collection and supporting information collected in these systems. I didn’t proactively volunteer or anything, I just used the Vsafe system and said I was pregnant around when I got my vaccine and was contacted.
posted by MadamM at 8:35 PM on June 22, 2022

The thing to remember is that there are hundreds, if not thousands of vaccines in active use. There are maybe fifteen that are controversial in the way that causes people to make fake reports. The tick-borne encephalitis vaccine is likely receiving unpolluted data even while the COVID vaccines are getting garbage.
posted by Tell Me No Lies at 10:50 PM on June 22, 2022 [1 favorite]

Also, they do things like follow up calls.

On one of my - 4-5 months after my Covid booster checks, I reported I'd had a day of bad health in the relevant period. I was really sure it was due to a new med change (and said so on the initial report) because a) I'd started a new med, it wasn't like the timing was mysterious to me and b) the info for that med says "You might have a bad day or so in this specific way" which was what happened for me.

I got a call a couple of weeks later, and when I called back, had a five minute conversation where they asked about that, I said "Oh, here's the situation" (started a new med, here's what it is, this is a pretty normal reaction, it's fine now.") and that was it. But I also appreciated them checking, and the way they framed the questions, which was pretty clearly designed specifically to help with cleaning up data irregularities if someone were playing the system.
posted by jenettsilver at 5:38 AM on June 23, 2022

Also, they do things like follow up calls.

On one of my - 4-5 months after my Covid booster checks, I reported I'd had a day of bad health in the relevant period. I was really sure it was due to a new med change (and said so on the initial report) because a) I'd started a new med, it wasn't like the timing was mysterious to me and b) the info for that med says "You might have a bad day or so in this specific way" which was what happened for me.

This is actually v-safe, not VAERS. v-Safe is an active, opt-in system (you say ok and CDC contacts you at various timepoints) specific for COVID-19 vaccines. VAERS is a passive system (you contact CDC) for all vaccines. I don't have any special knowledge of v-safe, but I imagine given the novelty and mass administration of COVID-19 vaccines, CDC wanted to collect these additional data. Neat!
posted by quadrilaterals at 6:41 AM on June 23, 2022 [5 favorites]