Upon this ROC I will build…

Upon this ROC I will build… April 1, 2012

I was considering writing an April Fools day post, but life’s too short to not blog about epidemiology and epistemology.

As is often the case when I go back to Yale for alumni debates, I ended up in some extended theological debates.  In a conversation with one friend, we ended up on thse topic that used to form the core of my about section: what evidence would persuade me that Christianity (or another religion) is true?

I think it’s possible that I’ve set the bar for proof so high that even a true religion couldn’t pass (it’s probably not a good sign when your possible proofs are physically impossible under current models of physics).  On the other hand, I certainly don’t want to keep my mind so open that my brain falls out.  But talking only in terms of these two extremes isn’t very helpful, so let’s all take a brief biostatistics detour, and see if it helps.  (It was extremely useful in my discussion this weekend).

When you’re evaluating a diagnostic test in epidemiology, there are two statistics you really care about:

  • Sensitivity – How often does our test successfully return positive results for true cases of the disease?  Sensitivity is calculated as (# of sick people correctly identified as sick)/(# of sick people total)
  • Specificity – How precise is our test?  Can we be confident a positive result really means the person is sick?  Specificity is calculated as (# of healthy people with a negative test result)/(# of healthy people total)

You can think of sensitivity as the true positive rate and specificity as the true negative rate.  In a perfect world, both statistics would be 100%, but that almost never happens.  We have to decide whether we care more about definitely finding every one who’s sick or if there’s a risk in casting our net too widely.  The TSA prefers high sensitivity screening, even if it’s a big inconvenience for non-terrorists.  When it comes to prostate cancer, we need better specificity, because treatment does more harm than good for a lot of people who test positive.

So what we do is move around our positive test cutoff to what seems like the optimal point.  To try and find that point, we draw a ROC curve.  (ROC stands for Reciever Operating Characteristic, but trust me, it’s not necessary to know what that means).  A ROC curve is a graph of the sensitivity and specificity (actually one minus the specificity) for a range of possible positive test cutoffs.  The curves look like this:

Real data is noisier than this

Usually you pick the cutoff for evidence that puts you at the ‘elbow’ — the point on the curve closest to the top left corner at (0,1), unless you’re in special circumstances like the TSA or prostate cancer oncologists.

So the question my friend and I were really kicking around is whether, when we consider religious propositions, should we stick to the standard way of thinking or is there a reason we should prioritize specificity over sensitivity or vice versa?

Generally, whenever we’re being asked to do something that clashes terribly with our moral intuitions (like, say, slaughter Isaac) it seems like it makes sense to stick with a high specificity test.  We’d rather miss a few genuine commandments and corrections in order to avoid hurting people by mistake.  But we don’t want such a low-sensitivity epistemology and standard of proof that we can never embrace a true philosophy and just stay stuck out on our own, trying to construct an entirely new edifice.

Epidemiologists have it easy.  They compute sensitivity and specificity by using a ‘gold-standard’ test (like autopsy) as a comparison to the field test.  I’m not really sure how to check the calibration of my sensitivity-specificity trade-off when it comes to ethics and philosophy.  Any suggestions?

"I'd love to see a video of how it works. keranique shampoo reviews"

Welcome Camels with Hammers to Patheos!
"Logismoi (the plural of logismos) are a fairly simple concept; they are whispers from either ..."

Logismoi, Vampires, and Other Intrusive Thoughts
"I imagine I’ll do a lot more reading and pick a lot more fights over ..."

A little about the queer stuff
"You are part of a search and rescue for lost Catholics.Regular updates to the countdown ..."

I’m keynoting at a Con for ..."

Browse Our Archives

What Are Your Thoughts?leave a comment
  • Patrick

    Its hard to come up with a evidential standard for what evidence would be necessary to accept a belief that’s been carefully customized so that its truth is as indistinguishable as possible from its falsehood. Just ask them how their belief pays rent. If the answer is “it doesn’t,” tell them that believing it is per se irrational because there are other mutually exclusive non rent paying beliefs out there, and choosing between them would be epistemically arbitrary. It saves time to get that out of the way right up front. Arguing about what evidential standard WOULD be necessary to convince you is a waste of time if they’ve already committed themselves to providing you no evidence whatsoever. It just gives them the rhetorical opportunity to make a bad faith accusation of bad faith.

  • Jerry

    That framing is correct if there is a path from reason to any “true” religion. Having lived the greatest portion of my life in evangelical protestant circles (trying to outlast that period now) I would say that most evangelicals, even the William Craig Lane’s of the world, would never expect to do more than show you that the philosophical basis of what they believe is not unreasonable – but even the non-existentialists would have some sort of leap of faith through some overwhelming personal “event”, like Lewis’ “surprised by joy”.

    Most would not even want you to come to the place where you were Roc curved into faith submission, as that leaves too much of the credit to you! Indeed, I think most Christians would be happy for you to remain at the place where there could be no evidence that would sway you, creating the exact atmosphere and circumstance that a Damascus road conversion requires.

    • Patrick

      William Lane Craig has argued, repeatedly, in public and in print, that theism is rational, and that atheism is irrational. Perhaps other evangelicals take the position you describe, but not him.

      • @b

        Rather than protestant evangelicals, the usual example for the strictly defensive position are the last scholarly theologians to exit academia proper.

  • Jerry

    read William Lane Craig there, sorry.

  • Joe Pickhardt

    I was having a conversation about this sort of thing last night too, actually. Hume, apparently, has a long discussion of whether or not we should believe in miracles set up in a similar way to what you’re suggesting.
    However, instead of approaching directly the question of, say, whether Jesus rose from the dead, he tries to understand the circumstances and evidence required for someone, in their everyday life, to believe someone telling them of an event that would seem to deny the laws of nature. What would it take for you to believe me telling you I flew yesterday? Probably not my testimony alone, nor that of 5 friends. There are very few circumstances under which you would believe something like this from testimony. Other factors, however, might make it more likely that you’ll believe. Maybe I have a video on my phone that shows me flying. You still might not believe me, but it would be much easier.
    The implications of setting up whether or not to believe in miracles this way, my friend is arguing in his senior essay, become very significant for christians — look at the actual evidence that someone has to say that Jesus rose from the dead.
    Apparently Hume ends this discussion by saying that it would basically take a miracle for any individual to believe in miracles, because the weight we can get from testimony is so small compared to our solid understanding of natural laws that continue to function. So all believers in Christianity, the implication is, all had miracles performed on themselves to make them believe in miracles.

    Or something like that; this was a short conversation over drinks. But it might be worth looking into.

    • Yes, this is a good summary of Hume. It has been around for a long time, in other words. And it only matters if you believe because you read it in the Bible. Right? But since reading the Bible is very rarely the inciting conversion experience–or, at least, there has to be a lot more to it–this doesn’t strike me as a problem. (It is a problem for anyone who is claiming that all one needs to do to believe is read the Bible, though. But that quite frankly seems like a foolish evangelical attempt to me, not least because it doesn’t actually understand the nature of conversion.)

      I should add that I have nothing but anecdotal evidence to back this. I would like to see a study on it, but that’s not the same thing as linking to a study on it.

      • Or, I suppose, it matters to those who believe because someone else claims to have seen the Virgin Mary or Jesus or whomever. Is that a thing that happens?

        • @b

          Yes because if the story-teller is revered then the story is profound. If they’re trusted, then it’s believable. “Mother is the name for God on the lips and hearts of all children” (The Crow)

          >>it would basically take a miracle for any individual to believe in miracles

          And yet forming/absorbing a mistaken belief is easy. Defending it in the face of conflicting teachings is too. The motto of the faithful is to believe in themself, to hope for deeper trust (Faith) in their existing religious beliefs about the God they have in mind.

  • Jack

    I think a maxim from Carl Sagan applies here: “Extraordinary claims require extraordinary evidence.” It’s not unreasonable to demand evidence incompatible with our current understanding of physics, if the claims a religious believer is trying to get you to believe are themselves incompatible with physics. So I’m happy in the low sensitivity, high specificity part of the ROC in any matters concerning supernatural incorporeal beings, life after death, and various other manifestations of magical thinking. More to the point, it’s reasonable to be in that part of the curve when dealing with religion.

  • Ray

    Well, the usual thing you’re supposed to do trying to distinguish between empirically identical claims is apply some form of Ockham’s razor, (which may be formalized by way of kolomogorov complexity, minimizing free parameters etc.) I’ll leave the reader to decide whether standard model + general relativity + inflationary boundary condition is simplified by adding the Athanasian creed into the mix.

    That said, I think this framing is giving up too much. It used to be that religious claims did have empirical consequences. Unfortunately, pretty much all of the empirically testable hypotheses inspired by religious claims turned out to be wrong (Circular planetary orbits, Geocentrism, Vitalism, End-of the-World Predictions, Flood Geology, Intercessory Prayer or shrines such as Lourdes as an effective medical treatment etc.) and the smart religious have become a bit gun-shy about making predictions. Nonetheless, it would still seem that religious belief still leads religious believers to make poorer predictions than religious non-believers on average (While there are believers at the highest levels of scientific achievement, the correlation between religious belief and scientific achievement is indisputably negative.)

    • deiseach

      “Intercessory Prayer or shrines such as Lourdes as an effective medical treatment”

      What? If you’re suggesting that, for instance, someone who is suffering from cancer is recommended to go to Lourdes instead of seeking medical treatment, I would agree with you that that is nonsense, but that’s not how it’s generally done. Either someone visits Lourdes when all avenues of treatment are exhausted, or it is a chronic long-standing condition – and there is no guarantee of a cure. Nobody, particularly the Church, will say “Everyone who visits Lourdes will be healed” or even “One person of all the thousands who go will be healed.”

      There may be a miracle, there may not. Miracles are not reproducible on demand (which is part of the whole problem with them from a material standpoint, yes?) Go to Lourdes expecting a physical cure and you will be disappointed; it is for spiritual benefit which is a different matter.

      • Ray

        So I can expect an official statement from the Church that the effect of going to Lourdes is indistinguishable from a placebo?

        As far as miracles not being reproducible on demand, neither are supernovae or earthquakes, and yet we can test all sorts of hypotheses about them. Perhaps the difference is that the one is a mirage that disappears when you look too closely, while the others are real phenomena.

    • Ted Seeber

      I find William of Ockham to be utterly unuseful. Most of the universe is more complex than the simple model will allow for.

      • Ray

        Perhaps you’d prefer the statement usually attributed to Einstein (http://quoteinvestigator.com/2011/05/13/einstein-simple/#more-2363 : As simple as possible, but no simpler.)

        Positing a simple explanation where a well supported theoretical framework leads you to expect a complicated explanation (e.g. in biology) does not make your overall worldview simpler. It is therefore not in accord with Ockham’s razor in the first place. Of course picking a random explanation of the appropriate complexity, without having the particulars forced upon you by the data isn’t a very effective approach either. So you basically just have to say, “something complicated is going on here and I don’t know what it is,” and leave it at that, or you have to collect a lot more data, in which case, once you’ve actually collected enough data, it’s very likely that the correct explanation is the simplest one compatible with your mountains of data. So the expected complexity tells you how much data you are likely to need before you can say anything interesting about what you’re trying to explain.

        Finally, in illustration of my point, I should also say that I have not given a complete description for how to do science, but rather a heuristic (so I’m not terribly interested in counterexamples.) There’s more than enough data to establish the fact that the human brain is a very complicated thing indeed, and it seems necessary for proper application of the scientific method, so it would be very surprising indeed if the method could be exhaustively described in a handful of unambiguous and exceptionless instructions.

  • Bdet

    The problem with atheists is that they are know-it-alls, but never used that “intelligence” to reach the conclusion that you are far from knowing any truth.

    • anodognosic

      Hi, Bdet, I’ve seen you here before making the same kind of comment: a short, sanctimonious drive-by that engages the issue tangentially if at all. I can’t speak for others, but for my part I think it’s obnoxious and disrespectful to a blog and readership that attempt to grapple honestly, intelligently and in good faith with opposing points of view.

      • Bdet

        It’s disrespectful based on what code of morality? That’s one thing I’ve never heard a solid answer about: if all that we are are just beings who are here to survive and conquer “weaker” beings, then why do we need this ability to debate? How has debate and the distinction between good and evil (or respectful and disrespectful) play a part in survival?

        • leahlibresco

          Hi, Bdet.

          Most atheists don’t think evolutionarily dominant strategies are a good guide to moral behavior. There are a variety of philosophers who have come up with systems of moral obligation that don’t hinge on the kind of God that Christianity proposes (Aristotle, Spinoza, and MacIntyre come to mind right off the bat).

          You can think that these alternative systems are built on sand (and I know plenty of Christians who do), but you probably won’t persuade anyone of that by just asserting that their moral system doesn’t work. You may have better luck if you ask, in a slightly different tone, what system they subscribe to, and then try and explore it from within and see where the weak points are.

          I think some of the commenters are interpreting you as hostile (and I have sympathy, since my debate background means I sometimes skip the pleasantries when I think I smell blood in the water), so remember that responses are voluntary, and it’s easier to keep people engaged if they don’t feel like your tone gives them an excuse to walk away.

          • Bdet

            I’m not really asking what moral school people subscribe to but more so, where does your understanding of the actual knowledge of the difference between good and evil come from? I don’t believe atheism has the answer for this.

          • Bdet

            And in this discussion, I’ve never asserted that anyone’s “moral system doesn’t work.” You must have misunderstood something I said. And yet, of the responses I’ve received, no one has bothered to really attempt to answer my question.

        • anodognosic

          Bdet, I’m not arguing metaethics with you*, I am pointing out that you are being disrespectful and obnoxious. I hope that you will try to participate more considerately in future, but in the meantime I’ll just take care not to feed the troll.

          * Although on that point, rest assured that you have misunderstood more than one aspect of contemporary mainstream atheist thought in your response, and on that count, you would do well to put in the time to actually read and understand the posts and discussions on this blog.

          • Bdet

            So you have yet to actually respond to my question.

          • Bdet

            You keep calling me disrespectful but I still don’t understand how you (or any human) came to the understanding of this morality. We are the only creatures who mind this aspect, and I believe that is a HUGE indicator to the fact that we are made in God’s image. But since you are refusing to answer me and call me a name instead, I’ll just conclude you don’t have the absolute truth as you my have thought you did.

  • Kyle

    Suggestions? Here’s mine: read G.K. Chesterton.

    “I am incurably convinced that the object of opening the mind, as of opening the mouth, is to shut it again on something solid.” -G.K. Chesterton

  • Andrew G.

    One word: “prior”.

  • “I’m not really sure how to check the calibration of my sensitivity-specificity trade-off when it comes to ethics and philosophy.”

    Yup. That seems to be a problem. This is a really attractive idea, I must say. It approaches the question honestly. Pascal, I think we know, would suggest that hellfire is good enough reason to choose sensitivity, but since you’ve said that you would be a universalist if you converted, hellfire would only be temporary. That nixes the nasty “infinite” from the negative and does away with Pascal’s whole set-up. (Yes, yes, this has been done before.)

    My point, though, is that this whole thing looks like Pascal’s wager, just dressed up. You assess whether you prefer sensitivity or specificity based on how bad you think a false alarm is compared to how bad you think a miss is. Right? (Honest question: am I following this correctly?) Missed terrorists are worse than harassed civilians; treating healthy prostates is worse than missing cancerous ones. Or at least so the logic of the TSA and the health services goes. So this fails (for me) in the same way that Pascal’s wager fails: we cannot know for sure what the costs and benefits are. We actually have no idea at all. (I’d say we don’t even know what the benefits of being right about atheism are. I’m not at all sold that an atheist’s life is any better, or an atheist society is any better, than a Christian version of the same, even if I assume that atheism is true. Specifically, I have seen no evidence for such a claim. But that’s another story.)

  • You assess whether you prefer sensitivity or specificity based on how bad you think a false alarm is compared to how bad you think a miss is…Missed terrorists are worse than harassed civilians; treating healthy prostates is worse than missing cancerous ones….So this fails (for me) in the same way that Pascal’s wager fails: we cannot know for sure what the costs and benefits are.

    I think this is a valid point. There’s no way to balance specificity and sensitivity without knowing the consequences of a “miss” in either direction. Even if you hold as possible the risk of eternal damnation (worst case scenario) a) that only gives the consequences of a “miss” in one direction and b) you have to take into consideration how likely you think that consequence is, in which case you’ve already in a sense invalidated the test by introducing subjective measures rather than objective ones, haven’t you?

    • anodognosic

      Delphi, I don’t quite understand how considering the probability of hell would push you into subjectivity. I’m an atheist, but I believe the probability of hell existing is far, far lower than that of God existing. Where exactly do you draw the line between objective and subjective in this situation?

      Also, Pascal’s wager is extremely problematic, because any claimed potential outcome that promises infinite reward or infinite punishment is going to short-circuit any good decision-making system, because on the face of it, the expected value for that outcome will overwhelm all others even if its probability is vanishingly small. Such claims, thus, have a high potential for manipulation, and should require a heightened scrutiny–that is, a high degree of specificity. If not, we will be constantly yanked about by spiritual snake-oil salesmen.

  • Kyle

    I like your analysis with the Isaac situation. Maybe your ROC curve has to adapt to other criteria, like the potential harm that could result to you or someone else.

    Here is my easy 3-step ethics and philosophy ROC adjustment algorithm:

    1. Suffer some false positives on things with a small potential for disaster.
    2. Suffer some misses on things with a high potential for disaster.
    3. Re-evaluate ROC curves based on results of steps 1 and 2 and repeat.

  • Hmm, I guess I can only say what I always say:

    If you believe in objective morals and don’t believe in any supernatural process that would have aligned your native culture to it, and your moral intuition was formed by that culture, then you must be near certain you presently support grave evil.

    By not having a metaphysical grounding for your ethics you are missing the only way to correct that. This should count as an Isaac argument against your present position and, to stay in your image, drive you pretty far up the ROC.

    • leahlibresco

      Yes. This is pretty much the biggest problem with my position. Since evolution is undirected, we lit on morality by chance. The best answer I can offer in an atheist framework is that intelligence and perception compels us to start understanding morality in the way it does physics.

  • Ted Seeber

    Here’s an entirely different approach. Look upon it not as a physics experiment, but as an experiment in anthropological evolution, with the goal being human peace in this world both individually and as a culture.

    Where this lead me was the search for the genetic roots of religious belief and towards religions that, through survival of the fittest, had created a morality that made rational sense. Which basically narrowed it down to Buddhism, various forms of Shammanism, and my own cradle Catholicism, though not for the reason you would suspect.

    All three of these are based not just in metaphysical explanations and revelation called “The Deposit of Faith” in Catholicism, but also in observation of natural law.

    I came to the conclusion that there are three ways to go- in tune with nature in accordance with one’s local environment and local climate (Shammanism in a myriad of local tribal forms). Acceptance of what we can’t change and adjusting our ideals to fit what already is (Buddhism). Or holding to ideals but realizing the human race is fallen and will never live up to those ideals, though we get closer with each passing generation (Catholicism).

    Note, I said Catholicism, not Christianity- the majority of Christian denominations are far too young to have enough observational experience, and tend to repeat the mistakes of the past. Catholicism is still a thousand years younger than Buddhism as well, and is working with significantly less data.

  • Looking at the old stuff here I’m curious about the lead-in sentence:

    I was considering writing an April Fools day post, but life’s too short to not blog about epidemiology and epistemology.

    Does this mean you actually had an April Fools day post all planned out and then it turned out to be true ? That would be awesome, but then sometimes a cigar uh intro sentence is just an intro sentence.

    • leahlibresco

      No, but there is a funny story about this post. Check the date (against the liturgical calendar). I switched sides the night before. This is the first post I wrote as a Christian (or aspiring Christian), and it did occur to me that if I had written up the events and conclusions of the previous night, it would have been assumed to be an April Fools day post.

  • Darren

    ”What evidence would persuade me that Christianity (or another religion) is true?”

    What evidence would persuade me that the world was actually a virtual reality simulation and I a sentient program within?