Flawed Paper on COVID-19 Vaccines, Deaths Spreads Widely Before Retraction

SciCheck Digest

The COVID-19 vaccines have been shown in trials and real-world application to be safe and effective. But a paper shared widely online claimed that vaccines cause two deaths for every three lives saved. Experts say the analysis misinterpreted data and was flawed — and it has now been retracted by the journal that published it.

How do we know vaccines are safe?

How effective are the COVID-19 vaccines?

---

Full Story

Unverified reports of adverse events that occur following receipt of a vaccine have been a key source of misinformation for months. As we’ve repeatedly documented, anyone can submit a report to the U.S. system and the reports alone are not proof of a link to a vaccine.

In the latest iteration of that type of data being misrepresented, a peer-reviewed paper published in a scientific journal used it to dubiously claim that the COVID-19 vaccines were causing deaths at such a scale that they nearly rival the number of lives saved through the vaccines.

“For three deaths prevented by vaccination we have to accept two inflicted by vaccination,” the paper, published on June 24 in Vaccines, claimed.

Days after the analysis was published — amid backlash from scientists, some of whom resigned from editorial roles with the publication — the journal issued a June 28 statement noting that there were “significant concerns” with the paper and that its assertion that the deaths were linked to vaccination “is incorrect and distorted.”

“The data has been misused because it makes the (incorrect) assumption that all deaths occurring post vaccination are caused by vaccination,” Katie Ewer, an immunologist at the University of Oxford, told Science in an email. “[And] it is now being used by anti-vaxxers and COVID-19-deniers as evidence that COVID-19 vaccines are not safe. [This] is grossly irresponsible, particularly for a journal specialising in vaccines.”

Vaccines, based in Switzerland, formally retracted the paper on July 2.

But in the days between its publication and retraction, the paper and its claims found a significant audience on social media, where they were spread widely across platforms.

The link to the paper was shared nearly 9,000 times on Facebook as of early July 2, according to CrowdTangle analytics data. Metrics on the journal’s site indicate it was viewed more than 367,000 times.

Liz Wheeler, a conservative commentator, promoted the paper in a June 30 video on Facebook — viewed more than 250,000 times — titled, “The Vaccine Study You’re Not Allowed To See.”

“This is a peer-reviewed, scientific study,” Wheeler tells her viewers. “Essentially, this peer-reviewed scientific study shows that COVID-19 vaccine causes two deaths for every three lives it saves.” 

Behind her in parts of the video is a board repeating the study’s claim as a statement of fact: “COVID vax kills two people for every three saved.”

As we said, the journal had acknowledged two days earlier that the data about purported deaths from the vaccines had been “distorted.”

The paper said that the authors used “the data of the Dutch national register … to gauge the number of severe and fatal side effects per 100,000 vaccinations.”

But the Dutch reporting system — maintained by the Netherlands Pharmacovigilance Centre, called Lareb — includes a prominent disclaimer explicitly stating that a reported side effect may not be from the vaccine, similar to the adverse event reporting system used in the U.S., which we’ve explained.

Dr. Eugène van Puijenbroek, head of science and research at Lareb, told us in an email that the data used in the analysis “were interpreted incorrectly and therefore led to erroneous conclusions.” He said he had requested that the journal “make either an appropriate correction” or retract the study.

The system is used to “highlight possible safety signals at an early stage,” he said. “A reported event that occurred after vaccination is therefore not necessarily being caused by the vaccination, although our data was presented as being causally related by the authors. It should be noted that also the database of the European Medicines Agency that is mentioned in this article contains the same type of reports.”

“As clearly stated on our website, but not taken [into] account in the method applied in this article, death after vaccination does not imply that the adverse event is indeed the actual cause of death,” he said. “Existing health problems turned out to be the most obvious explanation for the fatal outcomes in the majority of cases. In a limited number of reports, adverse events following immunisation, known to be associated with the COVID vaccines (e.g. fever, nausea or malaise), may have contributed to the worsening of an already fragile health condition or an underlying medical condition. Suggesting all reports with a fatal outcome to be causally related is far from truth.”

Also, Van Puijenbroek said, the paper’s assertion that the reported deaths “were certified by medical specialists” is “simply incorrect.”

The lead author of the study, Harald Walach, is a psychologist and professor at Poznan University of the Medical Sciences in Poland. He and his co-authors penned a two-page letter defending their study, published by Retraction Watch, in which they admitted “that these data are far from perfect” but said they were justified in using the data for analysis because “they are the only ones that are available.”

They also claimed that “[c]urrently we only have association, we agree, and we never said anything else.”

There were other issues with the paper, too.

In addition to the fact that the deaths and other adverse events reported to the Dutch system are not confirmed to be caused by the vaccines, the authors arbitrarily chose to use that system to look at the case reports submitted per 100,000 vaccinations. But that system’s volume is an outlier, relative to the rest of Europe.

“They have a median of 127 per 100k, ranging from 15 to 701, with Netherlands’ 701 being an extreme outlier,” Jeffrey Morris, director of the division of biostatistics at the University of Pennsylvania, told us in an email. “They give some unconvincing rationale why they think using the 701 to represent the truth is the best idea, thus strongly skewing the adverse event rate of vaccines by a factor of almost 6 relative to using the median for all of Europe.”

The paper also uses a concept known as the number needed to vaccinate, or NNTV, to estimate how many doses would need to be administered in order to avert one COVID-19 case or death. For deaths, it concludes that “the NNTV to prevent one death is between 9000 and 50,000 (95% confidence interval), with 16,000 as a point estimate.”

But using that kind of measure — also referred to as the number needed to treat, or NNT — is problematic for several reasons, Morris said, especially for something where the risk is spread over a long period of time.

“The key factor here is that NNT depends directly on the time frame, and when you compute it using data from a short term study, it severely [underestimates] the benefit when the risk is spread over a longer time frame,” he said.

The authors used data from an Israeli study that followed up with participants who received the Pfizer/BioNTech vaccine for an average of 15 days.

Morris said the short time frame for computing vaccine benefit is “especially misleading” since the reported adverse events (which again, aren’t confirmed to be due to the vaccine) are typically occurring within that short time frame — “but only a small fraction of the beneficial effects of the vaccine, preventing cases for the next year or two or longer, are captured.”

Also, the measure is influenced by other factors — such as lockdowns, social distancing and masking.

And when many people are vaccinated, the probability of infection and death goes down — therefore making the number needed to vaccinate higher (and the vaccine benefit appear smaller).

“In short, the NNT is an extremely flawed measure even if they had good data for estimating the risk of infection, hospitalization and death for vaccinated and unvaccinated groups and interpreted the data properly (which of course they did not),” Morris said.