Rick Santorum made two false claims regarding the scientific consensus on climate change:
- He said 57 percent of climate scientists “don’t buy off on the idea that CO2 is the knob that’s turning the climate.” This is based on a flawed reading of a survey of 1,868 scientists; one of that survey’s authors has called Santorum’s claim “absolutely false.”
- He called the oft-cited 97 percent consensus figure “bogus,” and said it was based on a survey of only 77 scientists. In fact, several surveys involving thousands of researchers have all found that the level of consensus that human activity is primarily responsible for warming is as high as 97 percent.
Survey Says …
Santorum, a former senator from Pennsylvania and 2016 Republican presidential candidate, appeared on Bill Maher’s show on HBO. Maher said that Santorum does not believe climate change is a real problem, to which Santorum responded:
Santorum, Aug. 28: And I’m not alone. The most recent survey of climate scientists said about 57 percent don’t agree with the idea that 95 percent of the change in the climate is being caused by CO2. … There was a survey done of 1,800 scientists, and 57 percent said they don’t buy off on the idea that CO2 is the knob that’s turning the climate. There’s hundreds of reasons the climate’s changing.
Santorum’s campaign did not respond to a request for clarification, but he is likely referring to a survey conducted in 2012 by researchers in the Netherlands and Australia. The results of that survey were published in 2014 in the journal Environmental Science & Technology, and a more thorough examination of the survey was published in April 2015 by the Netherlands’ Environmental Assessment Agency.
The numbers Santorum cited come from an analysis of this survey by bloggers at Fabius Maximus, a website whose authors include several retired military personnel, others with experience in finance and anonymous contributors. Santorum misstated the bloggers’ analysis, though — understandably, because this is confusing. The bloggers claimed that 57 percent of climate scientists disagreed with the United Nations’ Intergovernmental Panel on Climate Change’s latest assessment that there is a 95 percent chance that humans have caused more than half of observed global warming.
Bart Verheggen, one of the authors of the survey in question, told Peter Sinclair of Climatecrocks.com in a videotaped interview that Santorum did not accurately represent his study’s findings:
Verheggen, Aug. 31: We concluded that the level of consensus is somewhere between 79 and 97 percent, depending on whether you take the whole group or whether you zoom into those with more expertise. …. Rick Santorum said that the percentage [who disagreed] was in excess of 50 percent, which is absolutely false and doesn’t correspond to our findings. … What we did find is the more expertise respondents had, the more they agreed with the human dominance of global warming.
To understand why the Fabius Maximus analysis and Santorum’s claim are wrong, it is helpful to understand precisely how that survey worked and what it found.
The authors sent emails to 7,555 individuals gathered from a few similar sources: those who had published papers or assessment reports that included the keywords “global warming” or “global climate change” during the period from 1991 to 2011, a separate database of actively publishing climate scientists, and a separate review of climate science papers from 2009 to 2011. A total of 6,550 people were successfully contacted, and 1,868 questionnaires were returned, resulting in a response rate of 29 percent.
The survey’s first question asked these scientists “[w]hat fraction of global warming since the mid-20th century can be attributed to human induced increases in atmospheric GHG [greenhouse gas] concentrations?” There were several possible answers, allowing respondents to choose a specific range of contribution to warming. The combined group that pegged the contribution as above 50 percent — meaning, greenhouse gases have accounted for more than half of the observed warming — was considered to agree with the consensus. A total of 1,231 people agreed with the consensus, or 65.9 percent of the 1,868 respondents.
But Verheggen said the authors found a consensus of between 79 percent and 97 percent. How did he arrive at that range? Two of the possible responses to the first question were “unknown” and “I don’t know,” which the authors called “undetermined” responses. A large number of people selected these options: 9.9 percent said “unknown,” and 8.8 percent said, “I don’t know.” But undetermined responses do not mean the respondents don’t believe humans are the primary driver of climate change. Pinpointing the specific amount of human contribution is a difficult task.
The study authors argue that these answers should not be included in the analysis of the consensus, resulting in Verheggen’s range. The lowest possible value after excluding undetermined responses was 79 percent (see table S3 of supplemental information), among 278 respondents who had published only zero to three papers on climate science. The highest possible value was 97 percent, among 142 respondents who were authors on the IPCC’s scientific report published in 2007. Among all 1,868 respondents, the rate was 84 percent that agreed with the consensus.
So how did Fabius Maximus writers arrive at 57 percent disagreeing with the consensus? The short answer is, by using faulty logic and inaccurate assumptions.
The bloggers combined the responses to that first question (question 1a) with a follow-up (question 1b) that asked: “What confidence level would you ascribe to your estimate that the anthropogenic GHG contribution is more than 50%/less than 50%?”
Of 1,222 scientists who responded to question 1a that the contribution exceeded 50 percent, 65.2 percent said it was either “virtually certain” or “extremely likely” that this was the case. The IPCC considers “virtually certain” to mean more than 99 percent probability, and “extremely likely” to mean more than 95 percent probability.
Fabius Maximus took that 65.2 percent of 1,222 — 797 individual respondents — and divided it into the full 1,868 survey cohort, to arrive at 43 percent who, they wrote, agree (and 57 percent who disagree) with the idea that it is “extremely likely” that humans have caused more than half the observed warming.
There are multiple problems with this method. First, Fabius Maximus did not exclude the undetermined responses, as the study authors did. Including those answers in the final percentage assumes that all those respondents do not think human emissions are the primary driver of climate change, when again, it could mean that they find it difficult to pinpoint the amount of warming GHGs have caused.
Second, the Fabius Maximus authors counted only those who answered “virtually certain” or “extremely likely” — arguing that the IPCC itself said it was “extremely likely” that GHGs have caused more than half the warming, so that should be considered the consensus.
But that ignores an important point: The survey was done in 2012, when the Fifth Assessment Report from IPCC had not yet been released.
At the time of the survey, the IPCC’s official position — as stated in the Fourth Assessment Report — was that it was “very likely” (greater than 90 percent probability) that GHGs had caused most of the warming. That assessment was only upgraded to “extremely likely” in the report released in 2013. Thus, respondents who said “very likely” — another 24.1 percent of 1,222 — also should have been included as part of the consensus.
And further, another question in the survey asked respondents to “characterize the contribution” of various factors to the observed warming since before the Industrial Revolution. More than 80 percent characterized greenhouse gases’ contribution as either “strong warming” (more than 60 percent of respondents) or “moderate warming,” and this rose to more than 90 percent when “slight warming” was included.
Clearly, 57 percent of climate scientists do not disagree that “CO2 is the knob that’s turning 
the climate,” as Santorum put it.
There is also evidence that the more expertise a scientist has in the field of climate science, the more likely he or she is to say humans are causing most of the observed warming. The authors divided the survey respondents into four groups based on how many peer-reviewed papers they had published in climate science fields. As the number of publications increased, so too did the agreement that human-caused GHG emissions have had a “strong warming” effect (see chart).
Is the 97 Percent ‘Bogus’?
Santorum’s second claim was in regard to perhaps the most commonly cited figure regarding scientific consensus of climate change, that 97 percent of scientists in the field agree that human activity is primarily responsible for the observed warming:
Santorum: Number two, the 97 percent figure that’s thrown around, the head of the UN, IPC [sic], said that number was pulled out of thin air. It was based on a survey of 77 scientists – not even 97 scientists responded to that survey. Let’s talk about facts: And the fact is, lots of things cause climate change. …
Maher: Ninety-seven percent of all scientists believe —
Santorum: It’s a bogus number.
Santorum’s claim that the 97 percent figure is a “bogus number” because it is based on only 77 scientists is wrong. Several surveys involving thousands of researchers have all found that the level of consensus is about 97 percent. (We also can find no specific quote attributed to the head of the Intergovernmental Panel on Climate Change, or IPCC, as Santorum claimed. We have asked his campaign for a source on that, and for any other comment on these issues, and we will update this post if we get a response.)
The 97 percent number comes from several distinct sources. The first was a 2009 survey published in the American Geophysical Union’s Eos magazine. A total of 3,146 Earth scientists responded to two questions regarding whether Earth’s temperature has “risen, fallen, or remained relatively constant” since before 1800, and whether human activity is a “significant contributing factor to changing mean global temperatures.” In the full study, 90 percent answered “risen,” and 82 percent said yes to the human impact question.
The authors drilled down to those with more expertise: 79 researchers listed climate science as their specific area of expertise and had published more than half their recent peer-reviewed papers on climate science topics. Of those, 96.2 percent said “risen” to the first question, and 97.4 percent of 77 of them who responded said “yes” to the second question (this is likely where Santorum’s claim regarding 77 scientists originates).
A year later, another study published in Proceedings of the National Academy of Sciences found a similar result. The authors in this case set a specific criterion for expertise: All included must have authored a minimum of 20 peer-reviewed publications on climate science. Among a group of 908 such researchers, evidence from the publications themselves “show[s] that 97-98% of the climate researchers most actively publishing in the field surveyed here support the tenets of [anthropogenic climate change] outlined by the Intergovernmental Panel on Climate Change.” The authors also pointed out that the “relative climate expertise” of researchers who are “unconvinced” by those tenets is “substantially below” that of those who are convinced.
Finally, a 2013 paper published in the journal Environmental Research Letters has garnered perhaps the most attention for its consensus findings. That paper analyzed a total of 11,944 journal article abstracts published from 1991 to 2011 that matched the search terms “global climate change” or “global warming.”
From that list of papers, the study authors picked out which ones actually expressed a position on anthropogenic — human-caused — global warming, and which ones took no position. A total of 4,014 papers (33.6 percent) took a position, and of those, 97.1 percent endorsed the idea that humans are causing global warming.
A second analysis in that same study asked 8,547 authors to rate their papers. Did they think their papers endorsed the consensus on warming? A total of 1,189 scientists responded, rating 2,142 individual papers. The results were similar to the first part of the study: 97.2 percent of the papers endorsed the consensus that humans are causing global warming.
The authors concluded: “The number of papers rejecting AGW [anthropogenic global warming] is a miniscule proportion of the published research, with the percentage slightly decreasing over time.”
The authors of the paper address the fact that a majority of papers — 66.4 percent — took no position. They note that “the fundamental science of AGW is no longer controversial among the publishing science community.” In other words, there is no need to state a position on a topic that has largely been settled; a paper on some specific aspect of evolutionary biology likely would not state support for Darwinian evolution by natural selection — that is settled science.
The authors of the Environmental Research Letters paper support this finding by pointing out that more than half of the papers in the second part of the study that were self-rated by their authors as endorsing the consensus did not express a position in the abstracts. In other words, even though their papers didn’t state an explicit position, most of those authors say their papers implicitly endorse human-caused warming.
The Problem with Surveys
Calculating a number of scientists in the world who agree on a certain thing is a difficult task, as challenges to some of these consensus studies show.
The complication comes from two sources: the wording of the questions and the people responding. The survey discussed above, involving more than 1,800 scientists, asked the respondents to describe their field of study. The answers ranged from “climate impacts” and “climate modeling” to more general or less related fields, such as “geology,” “computer science” and “solar physics.”
This is why the separation of respondents based on the number of global-warming-related papers published produces a more valid result: The more work a scientist has done on a topic, the more we can expect that scientist to understand the field of study. A computer scientist who published a single paper on some aspect of climate modeling 10 years ago clearly does not have the same level of expertise as a scientist who has published dozens of papers on how the climate is changing.
As Verheggen said, with increasing expertise comes an increasing consensus that humans are the dominant force behind the changing climate.
The wording of the questions is also at issue in this study. A survey would produce very different numbers if it simply asked: “Are humans the primary cause of climate change?” Instead, this survey posed a nuanced question with many possible answers about the degree to which humans are causing the climate to change, and how incrementally certain one is of that human influence. It is easier to distort a study with several possible answers than it is to twist a study with “yes” or “no” questions.
According to both a careful reading of the study and to the author himself, though, Santorum is not correct that 57 percent of climate scientists don’t agree with the prevailing consensus on climate change. There is, in fact, a fairly large consensus — as high as 97 percent based upon multiple studies of varying size, composition and method — that human emissions have been the primary driving force behind observed changes to the climate.
Editor’s Note: SciCheck is made possible by a grant from the Stanton Foundation.
– Dave Levitan