Yes, Scientific Consensus Really Does Matter

Yes, Scientific Consensus Really Does Matter September 13, 2017

In the comments of my last contribution, most of the discussion was about the consensus of climate scientists (even though that wasn’t what the post was about). Perhaps this is a good opportunity to clear this up and discuss the importance of scientific consensus.


Skeptics are perhaps hesitant to use the heuristic of consensus to try and navigate which scientific beliefs are most likely true or not. We should absolutely be strong, independent, critical thinkers. And, to be clear, it’s a scientist’s job to avoid social pressure in the process of producing valuable data, since science is set up to eliminate as many biases as possible. You could say that science is founded on bucking the consensus. Richard Feynman famously spoke to high-school teachers in 1966 and produced this invaluable quote:


“Science is the belief in the ignorance of experts”


Science is not founded on consensus, obviously. Science is about making tests and looking at the data and taking them seriously, whether or not they align with what we already conceive as true.


That being said, science is a process, and not the facts that come out of science. Once the experiments have been done and the papers have been published and the data have been disseminated, we still need to do something with what we have. We still need to decide what is true, and base our actions on where the data lead. Papers can give conflicting results, the methods that the scientists used are often arcane and far beyond the realm of even perhaps a college-educated citizen. How do we determine what is true based on scientific methodologies?


Let’s define what a scientific consensus entails. When we discuss scientific consensus, we think of a large group of scientists who overwhelmingly agree on something. There may be some dissenting voices, but this is unavoidable with essentially any large group of people. You have a large group of experts who have spent a significant amount of time and effort looking at similar problems and arriving at a mostly similar conclusion. We may hesitate to go along with this consensus, because we don’t want to appeal to popularity. We know that humans are fallible, and can have faulty biases, which are not only present in groups but can possibly be amplified. Therefore, we should be avoid looking at what people say and focus on something else, to avoid the human element altogether.


Since we want to remove the human element from our search for scientific truth, let’s look at scientific consensus not as the aggregate of scientists’ opinions, but rather the aggregate direction of what the studies say. The studies are the studies, after all, there can’t be spin there. They just present the data and what the data say, so we avoid the problem of bias, right?


It turns out that this doesn’t exactly solve our problem either. Studies and data are inextricably linked to human behavior from beginning to the end of the scientific process. Humans are the ones that develop hypotheses, humans develop experiments and models and simulations, humans interpret the data and the math, and they discuss it in their conclusions. Not only this, but humans are the ones that created peer review have to meticulously pick apart every piece of a study, and evaluate these components to determine if they are worthwhile. Once published, humans read the studies, and cite them. We would like for studies to be cold and calculated and completely absent of human bias, but that is a red herring. Science is certainly more rigorous and as good at eliminating bias than practically any other aspect of human behavior, but it’s impossible to eliminate that factor altogether.


In fact, while the opinions of scientists will never be free from bias and hubris, their aggregate opinions should closely map to what the best papers and studies say. And that’s a good thing, because as laypeople (we are all laypeople save for perhaps a small handful of things), we may have trouble actually ascertaining what these studies say.


Are we as laypeople willing to claim we understand the ins and outs of every study thoroughly enough to make an informed opinion? Being well-versed in scientific knowledge requires years of study and poring over pages upon pages of scientific literature. It requires knowing the weaknesses of certain techniques and the strengths of others. For even a small subset of a research area, this can be a tremendous task. Are we going to lay claim that we can understand findings for things we haven’t spent much time on? It makes sense that we should give a high weight towards the opinions of experts over our own opinions and stances when it comes to things we are very poorly informed about.


One commenter on that post felt it necessary to ask how we know who an expert is and who isn’t, and took me to task. Truth be told, there isn’t a firm dividing line between who is an expert and non-expert, there is a sliding scale. As a basic operative definition, we know someone is an expert when they have put a great deal of time and effort into understanding something. This isn’t the entirety of expertise, obviously, but it’s something to go off of. This is also far from a perfect definition or metric. Someone could spend years on Google University™ looking up climate denial propaganda on the web, but we wouldn’t call them a climate expert. Someone could also put a lot of effort into trying to turn lead into gold, but unless they have a particle accelerator we wouldn’t exactly call them an expert in chemistry or atomic structure.


We can be fairly confident that the people publishing from Universities and National Laboratories fit these particular criteria for expertise. Every aspect of becoming a distinguished researcher is a grueling process. You are put through rigorous oral exams and required to demonstrate an in-depth area of knowledge. You are put under scrutiny by other well-established researchers who had to go through the same process many years ago. You have to publish papers regularly and file for grants showing that you have made significant discoveries, all the while having your work teased apart with a fine-toothed comb. While the process is far from perfect (I talked with Thomas Smith on his podcast about that), it’s impossible to make the case that it is easy or anything but meticulous.


It is these people who we should want to tease apart the data, and not us. We do not want to fall to the Dunning-Kruger effect, where we are so out-of-depth in a field of research that we can’t even evaluate our own incompetence. We also shouldn’t trust ourselves when we find research that aligns with our own preconceived notions without pausing to consider whether that research is fringe or not. If skepticism is about seeking the truth and eliminating our own biases, shouldn’t we filter our opinions through those that are most informed, rather than trying to reinvent the wheel and navigate unfamiliar scientific territory on our own?


After all, seeking truth is a collective effort. No one of us can become an expert in everything and none of us will know all that there is to know (a fact that I find more and more bothersome every day). Nobody has the time nor the resources, and certainly not the brain capacity, to have mankind’s cumulative breadth of knowledge. It makes sense that we allocate our knowledge and expertise to certain groups of people, and many groups of us try and piece together the whole puzzle to make everything fit together. If we don’t trust that peer-review and the scientific process is a useful pathway to truth, then science is the biggest conspiracy in the world and all the scientists are in on it. By contrast, it’s simply the most parsimonious explanation that most scientists have a generally accurate view of their field of study, and have a general understanding of the best findings and where most of the evidence points.


We don’t want to be the non-expert who thinks they know what they’re talking about, but simply has enough bad information to be completely off base. How many of us skeptics have talked to people who think the double-slit experiment implies that fundamental particles are part of a divine consciousness? Or how many of us have talked to people who think vaccines cause autism based on a retracted medical study? Just enough bad information can go a long way into leading us into a world of misinformation. Are we going to let our hubris as individuals try and trump the depth of information that others have put a lot of work into just because we think “thinking for ourselves” means ignoring what a lot of people have to say? Can we trust ourselves not to cherrypick our own sources? I’m not so sure that we can.


Furthermore, replication is also at the heart of scientific findings. One study could easily have faulty data or poor methodologies that don’t really bear fruit when another lab gets involved. Do you as a layperson have access to gaining scientific knowledge based on replication? I can safely say that most of the time you wouldn’t. You probably don’t truly understand the foundations of some of the equations published in a paper, or how one computational climate model differs from another and which techniques have certain flaws. Replication is one of the cornerstones of scientific findings, and you really do need to be an expert to be involved in this component. The only way you could know if a certain technique or finding is truly viable is if you had performed the technique yourself, or become familiar with enough literature that had reproduced that technique sufficiently.


I should finish off by saying that of course consensus shouldn’t be the end-all, be-all of any argument. We should aim to make sure we are informed ourselves about the best type of evidence, including what the studies say. Let’s be real here, this was inspired a climate change post and we know that consensus is a huge part of the argument about climate change. We know that atmospheric carbon has skyrocketed since the beginning of the industrial revolution, and we know from extremely basic chemistry that carbon emissions absorb wavelengths of light associated with heat. We also know from extremely basic chemistry that combustion reactions produce carbon emissions, and these emissions have to diffuse somewhere. We know that the temperature has shot upward in the past century, and that we can project that we are likely to see a 2-4 oC temperature increase by 2100 resulting in a sea level increase ranging from 0.2 to 2 meters. These are the better arguments to make rather than appealing to consensus, and we are always better off knowing the facts behind what the scientists think rather than just what they think.


However, we cannot ignore this useful heuristic. Under a Bayesian prediction, if nine out of ten dentists tell you that you have a cavity, are you more or less likely to have a cavity? If nine out of ten doctors tell you that you have cancer, do you seek treatment? If a survey shows that 97 out of 100 actively publishing climate scientists state that global warming is occurring, what do you take from it? You can cite “studies” like the Heartland Institute’s report that attempts to debunk climate studies, but these are written by people who are decidedly non-experts in the field. Why do you trust their methodologies more than the researchers? It’s good to think for yourself, but why would you as an individual critical thinker weigh this booklet against the piles and piles of literature that disagree with it? Why would you think that a non peer-reviewed booklet put out by a conservative think tank has better research methods than a group of scientists who live and breathe these data for a living?


Ultimately, we can’t ignore the consensus. Scientific consensus is a mildly flawed, but useful tool in informing us on the state of scientific findings. When our opinions are not aligned with the consensus, we should be very cautious. Perhaps we should see if we can get in touch with people who are experts, who can explain to us the basics, and perhaps if we’re getting it wrong they can educate us. A lot of our perception of our general state of knowledge would be solved if we were more transparent about what scientists think. Perhaps it would be better if more scientists communicated with the public. Even better, the media should focus on the direct opinions and findings of scientists, instead of focusing on politicians, pundits, and celebrities. Seeking truth is a collective enterprise, and we if we are truly critical thinkers, we should be doing our best at listening to those with relevant experience and learn what we have to say, since we will certainly never attain that level of experience for ourselves. Consensus may just be a heuristic, but it’s a useful one, and we would be foolish to dismiss it.

"Yes, biology and its stunning elaboration of systems, the brain and its numerous parallel systems, ..."

Idealism Is More Parsimonious?
"I’ve heard a variation of that, I think attributed to CS Lewis, that those who ..."

Lies. Liars and Truthtellers
"LB - I have no ideano truer Breuer words have ever been spokenplease show that ..."

The Most Pathetic Christian Argument I ..."
"No, what you just said seems rather completely unlike what I just said; particularly in ..."

Materialism As a Position of Faith

Browse Our Archives