In a hyped press briefing the eve of the Republican National Convention, President Donald Trump falsely said that convalescent plasma had been “proven to reduce mortality by 35%,” even though the therapy has not yet been shown to be effective for COVID-19.
Food and Drug Administration Commissioner Dr. Stephen Hahn also mischaracterized the data, inaccurately saying that a relative mortality decrease of 35% between those given high- versus low-concentration plasma meant that administering plasma would save 35 out 100 people sickened with COVID-19. That’s not right, and exaggerates the observed effect.
The administration’s inaccurate and imprecise summary of the existing data on convalescent plasma occurred in a Sunday night press briefing that Press Secretary Kayleigh McEnany billed as revealing “a major therapeutic breakthrough” for the coronavirus.
The press conference was held one day after Trump alleged, baselessly, that the “deep state, or whoever, over at the FDA” was making it hard for companies to test COVID-19 vaccines and therapeutics in an effort to delay results until after the Nov. 3 election. There is no evidence for such a claim.
Convalescent plasma, which dates back to the 1918 influenza pandemic, refers to isolating the liquid portion of blood from patients who have recovered from COVID-19, which may contain antibodies that could help sick patients better fight off the virus.
In announcing that the FDA was issuing an emergency use authorization for convalescent plasma to treat COVID-19 — a decision that days before, the New York Times reported had been put on hold after other federal health officials thought the data was too weak — Trump claimed that the therapy had “an incredible rate of success” and that a Mayo Clinic study had “proven” that it worked.
Trump, Aug. 23: We’ve provided $48 million to fund the Mayo Clinic study that tested the efficacy of convalescent plasma for patients with the virus. Through this study, over 100,000 Americans have already enrolled to receive this treatment, and it has proven to reduce mortality by 35%. …The FDA, MIT, Harvard and Mount Sinai hospital have also found convalescent plasma to be a very effective method of fighting this horrible disease. Based on the science and the data, the FDA has made the independent determination that the treatment is safe and very effective.
In fact, the FDA did not determine that convalescent plasma is “safe and very effective,” as few randomized controlled trials exist, making its true efficacy unknown. Instead, the agency decided that the totality of the evidence supports the idea that plasma “may be effective in the treatment of hospitalized patients with COVID-19,” and that it is reasonable to think that the potential benefits outweigh the known and potential risks.
Those risks include “allergic reactions, transfusion-associated circulatory overload, and transfusion associated lung injury, as well as the potential for transfusion-transmitted infections,” the FDA said in a press release announcing the EUA.
The FDA’s clinical memorandum emphasizes that additional research is still needed, writing, “Adequate and well-controlled randomized trials remain necessary for a definitive demonstration of [COVID-19 convalescent plasma] efficacy and to determine the optimal product attributes and appropriate patient populations for its use.”
The memo further explains that there are public results for only two randomized controlled trials of convalescent plasma for COVID-19 — one in Wuhan, China, and another in the Netherlands. Neither identified a statistically significant clinical improvement, although both trials were small and terminated early, and therefore may have missed the opportunity to detect a meaningful clinical benefit.
Previously, patients could obtain convalescent plasma through one of three pathways, including the expanded access program, which is what led to the Mayo Clinic study. Holly Fernandez Lynch, a professor of medical ethics at the University of Pennsylvania, told us that with the EUA, hospitalized patients can now receive plasma outside of those programs — “just like any other prescription drug, so long as it is provided within the terms of the EUA.”
Initially in the briefing, Hahn was more circumspect in his description of the data on convalescent plasma, saying that agency scientists concluded that “COVID-19 convalescent plasma is safe and shows promising efficacy.”
But in explaining the Mayo Clinic results, he neglected to mention that the study, which has not yet been peer-reviewed, was not a randomized controlled trial comparing treatment to a placebo — and that the so-called 35% mortality benefit was in a subset of patients, comparing those who received plasma with a high concentration of antibodies to those who received a low concentration.
“In the optimal treatment — the optimal patients … treated with convalescent plasma at the highest titers,” Hahn said, referring to the concentration of antibodies in the plasma, “there was a 35% improvement in survival, which is a significant clinical benefit.”
He then proceeded to incorrectly cast the finding as an absolute mortality benefit.
“So let me just put this in perspective. Many of you know I was a cancer doctor before I became FDA commissioner, and a 35% improvement in survival is a pretty substantial clinical benefit,” Hahn continued. “What that means is — and if the data continue to pan out — 100 people who are sick with COVID-19, 35 would have been saved because of the administration of plasma.”
When pressed by a reporter and asked directly whether he was saying convalescent plasma was a promising treatment or had been shown to be effective, Hahn dodged the question — and let the misleading interpretation of the study stand.
“So I would say that a 35 — if you’re one of those 35 out of 100 people who these data suggest or show survive as a result of it,” he said, “this is pretty significant for that person and their family.”
As we’ll explain, the 35% figure is a relative number, reflecting how much better patients who were given high titer plasma fared compared with those given low titer plasma. It does not mean that if the results hold that 35 people out of 100 would be saved by plasma. That would be an absolute mortality benefit, or the proportion of patients who avoid death because of the therapy.
In this case, the study did not test plasma against a placebo, so it cannot provide that information, although the absolute mortality reduction of giving concentrated plasma rather than low-concentration plasma, at least as reported in the preprint, is just 4.8% — meaning that fewer than five extra patients in 100 would survive as a result.
But even more critically, that particular result was drawn from a subset of participants and does not come from a randomized controlled trial, so it’s not definitive — and doesn’t settle the question of whether convalescent plasma is effective for COVID-19.
Hahn has since apologized for his flawed description of the Mayo Clinic results.
“I have been criticized for remarks I made Sunday night about the benefits of convalescent plasma,” he said in an Aug. 24 tweet. “The criticism is entirely justified. What I should have said better is that the data show a relative risk reduction not an absolute risk reduction.”
And in an Aug. 25 interview on “CBS This Morning,” Hahn said, “I personally could have done a better job and should have done a better job at that press conference explaining what the data show regarding convalescent plasma.”
Mayo Clinic Study
The Mayo Clinic study reports the results of more than 35,000 patients in the U.S.’s Expanded Access Program for convalescent plasma. Rather than assigning people to either receive plasma or not, as would happen in a randomized placebo-controlled trial, the investigators followed the outcomes of patients hospitalized with COVID-19 who were given plasma at nearly 2,000 sites across the country.
To get a sense of whether the treatment worked, researchers then analyzed the data to see whether patients were less likely to die if they received the plasma earlier in their disease course, or, in a subset of patients for whom their plasma was evaluated for antibodies, whether those who got plasma that was rich in antibodies survived better than those who received less concentrated plasma.
Overall, the data point to convalescent plasma being beneficial for COVID-19. According to the preprint, 8.7% of patients who received plasma within three days of diagnosis died within seven days, versus 11.9% of those who got it after four days or more. Similarly, the seven-day mortality rate was 8.9% for those who received the most concentrated plasma, compared with 13.7% for those who received the least concentrated plasma. That’s an absolute risk reduction of 4.8% and a 35% reduction in the relative mortality.
Still, as a Johns Hopkins University review notes, because of the study design, it’s not possible to tell from this work whether or not convalescent plasma is effective for COVID-19.
“[T]he cause of these differences in mortality may not be due to plasma therapy as the patients treated changed over time – were younger, healthier, less likely to have complications, and more likely to receive remdesivir later in the study – and the titer of antibodies transfused increased and the time to treatment also decreased over time,” the review reads. “This study highlights the need for randomized controlled studies to determine if there is a benefit for convalescent plasma for COVID-19.”
Mysterious 35% Relative Mortality Reduction
Trump and Hahn have emphasized the 35% mortality reduction from the Mayo Clinic study, but it’s worth noting that it’s based on a subset of patients. And it’s not even entirely clear which data produced the statistic.
As noted above, there is a 35% crude relative mortality reduction reported in the preprint at seven days for those who received high-concentration plasma versus low-concentration plasma. This number is based on 1,076 patients who received plasma that had been measured for antibody titers and met the cutoff parameters (see table 1).
But the preprint also shows a 35% relative decrease in mortality for high versus low concentration plasma at day seven when adjusted for potential confounding factors using a pooled relative risk estimate. That number is based on a total of 1,058 patients, approximately a third of whom were on a ventilator and a fifth of whom had “many” severe risk factors (see figure 3).
Previously, however, the FDA told the New York Times that the 35% statistic was based on other data from the Mayo Clinic study that the agency included in its EUA application — specifically a survival graph at 30 days for a subset of fewer than 1,300 patients who were younger than 80, were not on a ventilator and had received plasma within three days (see figure 3 on page 12).
Although the 35% is never written anywhere in the EUA application — and cannot be directly calculated because no exact numbers are provided — the Times estimated a 76% survival probability for patients receiving high-concentration plasma and a 63% survival probability for patients receiving low-concentration plasma from the graph, which works out to a 35% relative decrease in mortality.
Notably, Dr. Arturo Casadevall of Johns Hopkins University, who was one of the senior authors of the Mayo Clinic study, told the New York Times that he didn’t know where the 35% came from.
When we asked the FDA which specific data were used to make the 35% claim, the agency pointed us to Hahn’s “CBS This Morning” interview and promised to provide further information “when it is available.” We will update the story if we hear back. At this point, however, it remains unclear which information was used to generate the administration’s primary talking point.
Hahn’s Error
Regardless of the exact source, Hahn’s presentation of a 35% absolute mortality reduction, rather than a relative one, is incorrect.
While this might seem to be a small error, Dr. David Boulware, an infectious disease specialist at the University of Minnesota, told us that it wasn’t.
“THIS IS A HUGE MISTAKE which is very basic,” he said in an email. “I want to give Dr. Steve Hahn the benefit of the doubt that he was speaking extemporaneously and flubbed something,” he added, but the difference in this case is “huge.”
Indeed, if it is based on the preprint figure, that would mean Hahn exaggerated the effect of convalescent plasma sevenfold.
Boulware said he suspected that compared to no plasma, patients getting convalescent plasma probably are better off — but this remains an unanswered question. So are other critical questions, such as which patients benefit, which is especially important to know in deciding how to allocate a limited supply.
The Expanded Access Program, Boulware said, “was meant to be a short term bridge back in April. Yet tens of thousands of patients later, we don’t know what the benefit of plasma is.”
As for the EUA, Boulware said it “may” be warranted, but the hype beforehand was not — and that Hahn’s portrayal of a 35% absolute reduction in mortality was “deceptively misleading.”
While the administration claims the EUA will expand access of convalescent plasma, Boulware said it changes little other than the paperwork. “Plasma remains a scarce resource,” he said. “We use all the plasma locally that is available.”
More worryingly, Boulware thinks the EUA will make completing the necessary clinical trials more difficult to do. “People will want plasma which ‘saves 35% of people’s lives,’” he said, “and not enroll in randomized trials” in which patients may not be assigned to get the treatment.
For his part, Casadevall, a senior author of the Mayo Clinic study, supports the EUA and believes it will help improve COVID-19 health inequities.
“Plasma was already available in well resourced hospitals but understaffed hospitals caring for [underserved] populations struggled with the paperwork under the prior regs,” he said in an Aug. 24 tweet. “EUA makes usage easier.”
Dr. Scott Gottlieb, the previous FDA commissioner under Trump, didn’t doubt that the EUA decision was science-based, but echoed the call for randomized controlled trials of convalescent plasma.
“Plasma may provide a benefit, and it could be meaningful for certain patients, but we need more evidence to prove it,” he said in a tweet. “The data FDA had supports an authorization for emergency use, where the standard is ‘may be effective’ but we need better studies to confirm preliminary findings.”
After reviewing the published and unpublished evidence that underpinned the FDA’s EUA decision, the National Institute of Health’s COVID-19 Treatment Guidelines Panel issued a statement on Sept. 1 emphasizing the lack of available evidence on convalescent plasma.
“There are currently no data from well-controlled, adequately powered randomized clinical trials that demonstrate the efficacy and safety of convalescent plasma for the treatment of COVID-19,” the statement reads, adding that the FDA’s analysis of a subset of patients from the Mayo Clinic found no overall benefit in survival at seven days among those given high- versus low-concentration plasma.
Rather, an effect was found only in patients who were not intubated, with 11% of patients given high-concentration plasma dying within a week compared with 14% of patients given low-concentration plasma — a 21% decrease in relative mortality.
“Although these data suggest that convalescent plasma with high antibody titers may be beneficial in nonintubated patients, uncertainty remains about the efficacy and safety of convalescent plasma due to the lack of a randomized control group and possible confounding,” the statement said. “Additionally, antibody levels in currently available COVID-19 convalescent plasma are highly variable, and assays to determine the effective antibody titers remain limited.”
The panel concluded that for now, there are “insufficient data to recommend either for or against the use of convalescent plasma for the treatment of COVID-19.”
Update, Aug. 27: We added more information about how the EUA for COVID-19 convalescent plasma changes how patients can access the therapy.
Update, Sept. 3: We added the NIH COVID-19 Treatment Guidelines Panel’s statement about the FDA’s EUA for convalescent plasma.
Editor’s Note: Please consider a donation to FactCheck.org. We do not accept advertising. We rely on grants and individual donations from people like you. Credit card donations may be made through our “Donate” page. If you prefer to give by check, send to: FactCheck.org, Annenberg Public Policy Center, 202 S. 36th St., Philadelphia, PA 19104.