Consensus is Part of the Scholarly Method

Consensus is Part of the Scholarly Method April 21, 2016

An article in Real Clear Science highlighted that consensus-building is part of the scientific method. Alex Bezerow writes:

I like to imagine the scientific method as resembling the solar system. The planets, traveling in perfect orbits, represent the pillars of the scientific method: Observations, hypotheses, predictions/experiments, and continuous refinements.

What holds all of this together — the inward tug of gravity in this analogy — is consensus. We often call it “theory,” but that’s just a different word for consensus. Every scientific field has a unifying theory: for biology, it is evolution; for chemistry, atomic theory; and for physics, quantum mechanics and general relativity. We could replace the word “theory” with “consensus,” and the meaning would essentially be the same.

Theories can, and do, fall apart. Phlogiston isn’t real. The Earth isn’t at the center of the universe. Maggots don’t spontaneously generate on rotting meat.

How did such theories collapse? … Observations no longer made sense. Predictions weren’t coming true. Hypotheses were shown to be wrong. Only a new consensus could pull it all back together again.

That’s how science works. The scientific method produces consensus. But, if enough contradictory evidence arises, the consensus falls apart. Eventually, it crumbles completely and is replaced by a new consensus. (For more on this concept, read Thomas Kuhn’s The Structure of Scientific Revolutions.)

In the meantime, for anybody who is curious about what the current consensus is in any given field, read a review article or meta-analysis. These articles combine the best, most up-to-date information, and they try to paint a picture about where the field is headed.

Click through to read the original in its entirety. While some details are different in the humanities, the principles and the main point hold true.


"You may have “read through it”, but i lived it. I followed every detail that ..."

Not Liberal, Just Literate
"Right-wing conspiracy? Guess you can’t do basic math, can you? Explain to me how steve ..."

Not Liberal, Just Literate
"If this means the Trump Evanglicals go to heaven and leave the rest of us ..."

Trump as Antichrist

Browse Our Archives

Follow Us!

TRENDING AT PATHEOS Progressive Christian
What Are Your Thoughts?leave a comment
  • Matthew Funke

    I’m not so sure that the basic structure holds in the humanities. If you have a theory in science — like the theory of universal gravitation — you have an explanatory framework applicable *everywhere in the universe* under the domain of the theory. If you have a theory in medieval literature, you have an educated guess about a handful of books that is generally much less responsible to the concept of falsifiability.

    That’s not to say that the humanities are unnecessary or trivial. But I think they tend to treat the exploration of their subject matter rather differently.

    • The same principle does apply. In fact, in the humanities consensus counts for rather more than in other types of discipline. In history, as I keep having to explain to STEM-types who insist on “proof” and feel free to reject any historical conclusion that is not “proven”, a historian can’t prove or demonstrate for certain what happened in the past. They can only analyse the traces that past events may leave behind and make what they feel is an argument to the best explanation about what they most likely indicate. And what measure can we use to show that such an argument has explanatory power? Consensus.

      That the scholar in question is convinced doesn’t count for much. That their argument passed peer review counts for something, but generally only shows that it is coherent and possible, not necessarily likely and convincing. What actually counts is the argument convincing a sufficient number of peers, after suitable scrutiny and critical analysis. Given the critical nature of academic discourse and the rivalries and competition that comes with that culture, if someone’s thesis convinces a critical mass of their peers, this goes a long way to meaning their argument probably has some genuine explanatory power.

      In the humanities, consensus means something significant.

      Of course other elements always come into play (theoretical fashions, political pressures etc.), but consensus means something. This is why the peddlers of fringe theories don’t get it when a consensus that is against their theses is pointed out to them and they usually cry “argumentum ad verecundiam“. It’s also why the Jesus Mythers have to resort to conspiracy theories about wicked iron-bound academic orthodoxy holding terrified closet academic Mythicists in its terrible thrall. But the fact is that if your idea has been around a while and the consensus is overwhelmingly against it, all things being equal, you probably need to get a new idea.

      • What confuses me about the mythicists, too, is how certain they are.

        I have tried to be as objective as possible when I encounter a new idea, so when mythicists started telling me their view, I read one book, and a bunch of articles supporting their claims.

        For the most part, at least the mythicists I read didn’t even get their basic ancient texts correctly!

        And then some of them claim that texts are all subjective anyway.

        I’m not a professional historian, but I do know literature, was a literature teacher and remedial reading teacher for many years. Some texts are ambiguous, but many aren’t.

        When they say stuff like that–that the texts are subjective and could mean anything–yet at the same time, they claim that they are sure that Jesus never existed, I give up.

        • It’s mainly the kookier ones who are sure. The smarter ones hedge by declaring themselves “Jesus agnostics”. If they were truly agnostic on the issue you’d think they give as much credence to historicity as to mythicism and so be unable to make a choice either way. But these “agnostics” are strangely strident about how historicity is all wrong and Mythicism is wonderful and all encompassing. Their so-called “agnosticism” does allieviate them from presenting any truly solid and parsimonious alternative to how the whole Jesus sect got started though, which means they can fall back on some hand-waving and vagueness about “Euhemerism” and “historicisation” of myths.

          • Pseudonym

            It’s mainly the kookier ones who are sure. The smarter ones hedge by declaring themselves “Jesus agnostics”.

            I’ve noticed that the really canny ones try to have it both ways. That’s why you see titles and subtitles like “It’s time to ponder whether a Jesus really existed”, or “Why we might have reason to doubt” on top of works which claim high degrees of certainty.

          • I imagine titles like Despite Being an Unemployed Blogger, I’m the Greatest Genius This Field Has Ever Seen and Have Single-handedly Shown that the Scholarly Consensus on About Seven Separate Points Is Totally Wrong and That Everyone Else in the Field is a Poopy-head And/Or Insane are hard to get past editors, even the ones at Prometheus or some private press vaguely associated with Sheffield University.

          • Pseudonym

            It also reminds me of the way that Vox Day fancies himself as a political philosopher, but only publishes through his own vanity press.

            (Sheffield Phoenix is a legit academic publisher, though, and the fact that they published Carrier’s book should be all the evidence that anyone needs that there is no grand conspiracy against mythicism.)

          • I realise that Sheffield Phoenix is legit. I was alluding to the slippery way Tricky Dicky has kept calling it “the publishing house of the University of Sheffield” when it is nothing of the sort. The guy can’t seem to help himself – everything he’s associated has to be inflated to the most ludicrous degree.

          • Pseudonym

            Oh, I didn’t know he did that!

          • Yep. This blog post from July 17 2013 originally described “Sheffield-Phoenix” as the publishing house of the University of Sheffield (UK)”. But when he began to get some heat on Twitter over this last year he quitely amended the 2013 post. So the “the” was changed to “a” and the “of” was changed to “at”. I guess he thought no-one would notice.

          • Matthew Green

            And he accuses you of being dishonest?

          • The irony goes to unplumbed depths.

      • Matthew Green

        After you’re done responding to Carrier’s latest bout of intellectual diarrhea, I have an idea that might interest you. I was thinking that you can take your review and responses to Fitzgerald as well as your blog posts on the historical Jesus and post them at “History for Atheists”. If you have the time, I’d love to see a discussion on the problem of contemporary sources for people like Jesus, Hannibal, etc.

        • I hope to have at least one article relevant to this topic up on my (neglected) new blog this weekend.

      • Matthew Funke

        I take your point. My critique didn’t focus on consensus at all, though, or even mention it, but on falsifiability.

        • Historical claims are falsifiable by the evidence. One cannot run experiments on an ancient Assyrian military siege, but one cannot run experiments on the historical transitions that led to whales taking the precise evolutionary forms they do, either. One can simply study the evidence and seek to explain it. Those aren’t mere educated guesses. They are attempts to make sense of large or small quantities of data in as precise and accurate way as possible, and that requires an extraordinary amount of expertise. Indeed, if I were feeling argumentative, I might suggest that it is harder to have to embody in one’s knowledge the fullest extent possible of knowledge of ancient texts and artifacts so as to correlate a particular piece of evidence with the rest of it, than to run a laboratory experiment that relies on getting the equipment and its settings right. But I think playing the one off the other is inherently misguided. The fact that history and the natural sciences are mostly different in certain aspects of their methods doesn’t make one more rigorous than the other – contrary to what young-earth creationists and mythicists will tell you.

          • Matthew Funke

            It’s true that one need not have a lab or available experiments in order to have a falsifiable body of knowledge. It’s also the case that history is not the only discipline in the humanities. (I’d also appreciate it if you didn’t lump me in with the young-Earth creationists and the mythicists, since I’m not even *trying* to argue something like “historical science isn’t science”. It’s such a deviation from anything I’ve said that it’s grossly unfair, not to mention irrelevant.)

            Different disciplines have different amounts of rigor, and are more or less responsible to falsifiability. My example, for instance, was literary criticism. As another example, Aristotle’s theory (in his “Poetics”) that tragedy is composed of six critical elements (plot, character, thought, diction, song, and spectacle) is a useful conversation tool, but I would contend that it’s not as rigorously falsifiable as scientific theory. Music theory is not as rigorously falsifiable as scientific theory. Application of rational choice theory to religion (as Stark and Bainbridge have done) is not as rigorously falsifiable as scientific theory.

            There seems to be a salient difference in how the terms are used in science and in broad areas of the humanities. (The fact that history is sometimes classified in modern academia as a social science should highlight this distinction.) This difference, in turn, seems to indicate that the structure is different.

          • Music theory is a study of the norms followed in what is, in essence, a musical language. When rules that languages follow are given, they are not the same kind of thing as the rules that objects follow in a vacuum. Physics, music, history, linguistics, and other fields all work with different things, and I don’t know of anyone nowadays who thinks that the key distinction among them, whether in terms of method or rigor, is “falsifiability.” Scientists and historians may both respond to counterevidence to a prevailing theory by finding an ad hoc explanation. In neither field is it prudent to cast aside an understanding that seems well-supported because of one anomalous piece of data.

          • Matthew Funke

            I agree with what you’ve said. However, falsifiability is a critical component of a scientific theory; at the same time, is *not* a critical component of theories used by broad swaths of the humanities. (In particular, a handful of contradictory examples do not undermine an exploration of norms. They do, however, undermine a scientific theory.)

            My pointing this out does not imply that falsifiability is the key distinction between the different “theories” these different areas of study employ. It does not imply that I think music, history, linguistics, and physics all deal with the same things. It merely points out that there *is* a distinction, and an important one, and that we cannot therefore draw a strong parallel between theories as science uses them and theories as much of the humanities uses them.

            One can apply a very scientific approach to historical matters, and the consensus is valuable in both fields for substantially the same reason and carries comparable intellectual weight. However, I would stop short of saying that this is true of the humanities *generally*, as the original article stated, because the function, structure, and critical elements of theories in many areas of the humanities are different from theories and how they are used in science/history.

          • arcseconds

            Falsifiability was in vogue in philosophy of science for a while back in the 60s and 70s, but it’s largely regarded as being of historical importance only now.

            Just to get a flavour of this, you can check out the Stanford Encyclopedia of Philosophy or the Internet Encyclopedia of Philosophy, and note that they have no article for ‘falsification’ or ‘falsifiability’. Searching gets you to other pages where it is mentioned as, at best, one option among many, but on the whole it gives the impression the value is more historical than anything else.

            Or search the journal Philosophy of Science, and see that the articles a search on ‘falsification’ returns are largely from the 1970s.

            Falsification was of course a key part of Karl Popper’s idea of how theories relate to evidence, and I don’t think anyone thinks the classic Popperian picture is adequate any more. One of Popper’s concerns was establishing a demarcation criterion between science and non-science (especially pseudoscience) which might by why McGrath is talking of the distinction among various disciplines. Concern with demarcation criteria also doesn’t have the importance in philosophy of science it once did.

            So I always find it a little odd when people continue to insist on falsification as being important. It’s as though one were having an informed discussion on human motivation, and then suddenly finding people talking about sublimating libido and Oedipus complexes.

            There are a lot of problems with Popper’s view, but to take falsifiability in particular, it’s worth discussing Newtonian physics. Here it’s important to distinguish:

            1) the Newtonian theory (the laws of motion plus the law of universal gravitation)

            2) Newtonian models (which involve say planets with particular locations, masses and velocity, and (initially, at any rate) only attempt to treat some of the interactions)

            3) the Newtonian strategy of model refinement.

            Of these we can say:

            1) Newtonian theory isn’t falsifiable at all. For a start, it can’t actually be directly tested, you have to build a model and test that. Deviations from modelled behaviour just shows that your model is inaccurate. And note that by definition in the theory, deviations from rectilinear motion entail the existence of a force. So all motions are explainable by the theory.

            2) Models are falsifiable, but ­— in the initial stages, at least — this was expected, and the initial models were already known to be ‘false’, in the sense that they were not expected to explain all the data.

            But it seems weird saying that a simplified model that only looks at the Earth-Moon interaction is ‘false’ because it doesn’t explain things it never attempted to explain, and the data fits the model to the extent that was expected.

            And the picture of ‘rejecting the (a?) theory’ seems odd here too. I don’t think anyone actually says “I reject the two-body modelling of the Earth-Moon system!”: it’s still regarded as an adequate model of an important and locally dominant interaction.

            3) When do you abandon a model-refinement strategy?

            Certainly not on the failure of one model when models are expected to fail.

            Normally the models advanced by incorporating known-about bodies and interactions that hadn’t been incorporated before, but when there wasn’t a plausible contender, they did successfully posit the existence of new planets, so quite unexpected behaviour had been overcome.

            It was the anomalous orbit of Mercury that is the famous example of an interaction that Newtonian physics failed at, but at what point do you give up?

            Historically this wasn’t a matter of deciding the theory was falsified due to being unable to account for an observation.

            And in principle at least new forces could be introduced to get the orbit right (I believe this has been done, in fact).

            It was abandoned when a better theory came along. And what makes it ‘better’ isn’t necessarily a simple matter. One can say that Einstein’s theory handles Mercury’s orbit elegantly, whereas Newtonian mechanics would require a lot of ad-hoc modifications to make it work, but again it’s not a simple matter of a theory just being shown to be false here. Ad hoc adjustments get made all the time to theories, and no-one thinks this means that they have to be abandoned.

            Kuhn doesn’t get everything right here either, but his picture of ‘normal science’ and ‘anomalies’ has more going for it than fasifiability. I think Lakatos’s notion of a research programme is a better working up of what we actually see in the history of science.

          • Matthew Funke

            It looks like I have a fair amount of reading to catch up on. Thank you for your reply, especially where you tease out a little detail.

  • arcseconds

    Kuhn is potentially a dangerous person to cite here, at least without caveats.

    He is often read as saying that science is a non-rational, cultural enterprise, where things that don’t fit in to the reigning paradigm get swept under the carpet.

    This might be an attractive perspective for some mythicists, and don’t some of them even claim to be able to smell a paradigm shift in the air?

  • Matthew Green

    Only a very tiny handful of mythicsts are worth taking seriously. The one I respect the most is Robert Price. I strongly disagree with his views, especially Jesus as fitting perfectly into the mold of the “Mythic Hero Archetype” but Price is a friendly fellow and has legitimate Ph.Ds from a credible university. The only other scholary mythicist that I know of is Carrier. Despite having an actual Ph.D. from a reputable university, he’s hard to take seriously given his unprofessional demeanor towards people. I will give him credit; at least he has his work published by an academic press. Price has had his work published by a serious publishing company. The others haven’t.
    Price, to his credit, is more professional than Carrier is. At least Price can respectfully disagree with people whereas Carrier seems genuinely offended that people don’t agree with him and don’t praise his unique brilliance. If you disagree with him, Carrier thinks you’re either ignorant, stupid, dishonest, or insane. This is why I have no respect for the man anymore. I can respectfully disagree with a mythicist and keep it very civil. The moment that mythcist goes around vilifying people, respect goes out the window.

    • Pseudonym

      especially Jesus as fitting perfectly into the mold of the “Mythic Hero Archetype”

      I find that one quite funny. Joseph Campbell himself went into quite a bit of detail pointing out how Jesus does not fit perfectly into the archetype!

      Some details fit and some do not, which is exactly what you’d expect if a), you buy the monomyth theory, and b) the process of “mythologisation” was incomplete at the time the gospels reached their final form.

    • psstein1

      I agree. I strongly disagree with Price, but I like how he approaches disagreement and dispute. He’s willing to give the other side a fair hearing.

      Price’s issue is that he’s overly enamored with the “History of Religions” school, and doesn’t want to let their work go. That’s a methodological mistake. Carrier, on the other hand, thinks everyone else is wrong and only he (who has no academic job and few publications) knows the truth.