Thoughts on the Ideological Turing Test

The ideological Turing test that was running at Unequally Yoked has concluded, and if you’re like me, you couldn’t wait to find out how we did. Well, the results are in: here’s the answer key (some real surprises there!), and here’s how the voting shook out for the atheist round and for the Christian round. Read those first, and then come back for my thoughts on the outcome.

First of all, I want to thank Leah for running this contest, and I especially want to commend her Christian participants. They did an outstanding job of posing as atheists, much better than I was expecting, and one or two of them had me completely fooled. I highly doubt that a random sample of Christians off the street would have done anywhere near as well. Not counting the one answer that I knew was mine, I got better than 50% right, but not much better – certainly not enough that I can plausibly claim my score wasn’t just due to chance. I’m working on a followup post where I’ll list how I voted on each entry and explain what my reasoning was.

But here’s what I found most astonishing: Three Christians got higher atheist scores (i.e., were judged more likely to be atheists) than any of the real atheists! Conversely, if you look at the five people with the lowest atheist scores, two of them actually were atheists.

I’ve got to conclude that there’s something else going on here, something I don’t fully understand. If the Christians did an absolutely flawless job at passing themselves off as atheists, then we should have expected equal scores between the real and the fake atheists, because they would have been indistinguishable. Instead, it seems as if some of the Christians were judged to be “even more atheist” than some of the real atheists. (I’m happy, at least, to see that out of all the real atheists, I got the second highest atheist score, behind only Leah. She’s good at this!)

The only explanation I can think of is that the Christian winners managed to distill all the diverse atheist viewpoints they’ve heard into a statistical composite – an “idealized” atheist viewpoint that would strike anyone who heard it as typical of an atheist. We real atheists, meanwhile, each have our own quirks and idiosyncrasies which make us diverge from that idealized average.

Something else I wanted to point out is that far more atheists than Christians participated in this contest. 1,133 total atheists voted, while only 123 total Christians did. I can’t help but think this is a clue to why the results came out the way they did. It’s plausible that there’s a self-selection effect at work: atheists, in general, tend to be more interested in learning about different belief systems than Christians are, but that also means that there’s more of a spread of ability among us. By contrast, fewer Christians make the effort to learn about atheism, but those who do are highly motivated, and thus likely to do better in a contest like this one. Of course, I welcome hearing alternative hypotheses.

As far as the Christian round goes, I’ve got to be just a bit smug: I got the fourth-highest Christian score, higher than 6 of the 8 real Christians. Out of all the atheists, I was the second most-convincing Christian. (Leah beat me again. Didn’t I tell you she was good at this? Now you know why she’s on my blogroll.)

Still, when all the numbers are taken into consideration, I think the Christians showed us up this time, and I give them full credit for that. Because of unusual cases like clergy who’ve lost their faith but are still in the pulpit, we know that atheists can successfully imitate believers, sometimes for months or years on end. But in this competition, we didn’t do as well. I don’t think it’s necessary to be able to convincingly imitate an alternate viewpoint to be justified in rejecting it, but even so, I expected our side to do better than it did. There are clearly some atheists out there who need to hit the books!

About Adam Lee

Adam Lee is an atheist writer and speaker living in New York City. His new novel, Broken Ring, is available in paperback and e-book. Read his full bio, or follow him on Twitter.

  • Reginald Selkirk

    But here’s what I found most astonishing: Three Christians got higher atheist scores (i.e., were judged more likely to be atheists) than any of the real atheists!

    Check back in 10 years, and see if they are still maintaining the cognitive dissonance.

  • Hibernia86

    I think the issue I had was that even if you have a specific belief system, that doesn’t mean you know it all as well as the experts (we know this from the fact that Atheists do better in quizzes about the Bible than most Christians do in the general population). So in this test, a Christian could look up Atheist arguments on the internet, convert them into an essay, and sound more convincing than an Atheist who was just casually interested in the subject. The same thing is true of the Christian side. As Leah stated, this was all just to get a conversation started, but if we wanted to do a better test, we should interview people in real time so that they don’t have time to look up arguments and are instead put on the spot for an immediate answer.

  • http://kagerato.net kagerato

    I’ll make a quick reference to my previous comment on this, since it’s attached to a much more obscurely titled post “Cast Your Ballots Now”:

    Comment on Methodology of Leah’s Ideological “Turing Test”

  • http://www.secularplanet.org Secular Planet

    It’s remarkable how well some of these Christians can present the atheistic viewpoint. It’s even more remarkable how utterly terrible their justifications for belief in Christianity are. If even these people can’t present halfway plausible reasons, that says something about religion.

  • Reginald Selkirk

    The successful pseudo-atheists seem to be pretty familiar with the atheist arguments. You would think then that they must have some successful counter-arguments which they invoke to justify their beliefs. And yet the counter-arguments I encounter are uniformly terrible.

  • Socius

    I think the question “What evidence or experience (if any) would cause you to believe in God?” was key to weeding out the fake atheists. I notice that about half the Christian “atheists” could not think of anything that would convince them of God’s existence, but NOT ONE of the real atheists made such a claim. That kind of thinking is intellectually dishonest but among believers it is widely thought to be a hallmark of atheist thought: the “smug atheist” with a “hardened heart”.

  • KL (Kat)

    @Secular Planet and Reginald,

    I’m the Christian who was voted both the most convincing atheist and the most convincing Christian (by Leah’s weighting system). If you’d like me to rebut my “own” arguments from my atheist entry, I’d be more than willing to do so.

  • http://www.daylightatheism.org Ebonmuse

    I would be interested to hear that, Kat. If it’s too long for a comment, I’d even consider making it a guest post.

  • Socius

    It looks like the Christian round version of the question above (“What evidence or experience (if any) would cause you to stop believing in God?”) is equally good for weeding out the fake Christians. Most of the atheists-playing-Christians claimed that nothing could cause them to lose faith in God, but only 1 Christian made such a claim.

    It seems both sides see the others as hard-heads who would deny the truth even if it were right in front of them. But reading the genuine responses shows that this doesn’t hold up. Perhaps this misunderstanding should be addressed…

  • arensb

    some of the Christians were judged to be “even more atheist” than some of the real atheists.

    You mean, they were more athy? Perhaps even, dare I say, athiest?

  • KL (Kat)

    @Ebon,

    Sure! I do suspect that it would be a little long for a comment (without spamming your combox anyway), so perhaps I could send it to you via email? I don’t want to pressure you into giving me a guest post, though — I certainly don’t expect to be given a [religious] platform on your blog. If you think it would be interesting to your audience, though, you’d be welcome to use it.

  • Charles Black

    @Ebon

    Have you ruled out the possibility that some of the Christians who participated were merely cultural Christians & some of the atheists may have just deconverted & therefore don’t know all of the arguments for non-religion?

  • http://www.daylightatheism.org Ebonmuse

    Kat – Yes, please; feel free to e-mail it to me. Trust me, I don’t feel at all pressured. :)

  • http://indiscriminatedust.blogspot.com Philboyd

    Interesting analysis, Socius. If I may share a little of my personal experience, discovering Slacktivist through Ebon’s blogroll began the slow process of teaching me that no, not all Christians are pigheaded or fraught with cognitive dissonance. Bravo to all the eloquent believers who explain their faith clearly, both in Leah’s test and elsewhere! While I may not agree with their theism, it’s nice to see that not only can people be good with God, but that some Christians can use their faith to inform and strengthen their goodness.

  • Kaelik

    @Charles Black

    “Have you ruled out the possibility that some of the Christians who participated were merely cultural Christians & some of the atheists may have just deconverted & therefore don’t know all of the arguments for non-religion?”

    I would say instead that obviously Christians who spend a fair amount of time reading atheist blogs, such as Leah’s, are most likely to be more familiar with atheist arguments than the subset of atheists who read atheists blogs.

    Self selecting for people who read atheist blogs naturally weeds out all the Falwells of the Christians, and even the common Christian, who hasn’t really thought about atheism a lot, whereas atheist blogs are the closest thing to Sunday Church as atheists get, and so there is no particular reason to believe that they would be particularly good about either distinguishing liars, or mimicking Christianity.

    What was “proved” if anything, is that Christians who spend a lot of time on atheist blogs are better at sounding like atheists than atheists who spend a lot of time on atheist blogs are at sounding Christian, and no one believes that any post is actually made by a Christian.

    Which is about as revolutionary as saying that people who like checkers and play a lot of chess are better at playing checkers and chess than people who like chess and play a lot of chess.

    Though my system of making my decision almost entirely on the convincingness of the morality question caused me to be quite accurate, significantly more so than the average. So I do have that to cling to.

  • Kaelik

    Looking back, Saying “I’m a Kantian” had me declare Kat as a pretender, and I think I specifically commented on Adam’s to the effect that:

    “[Even though I generally used 'we all have the same morals' answers as a basis for declaring Christianity] this post did so in a particularly atheist way, so even thought it’s stupid, it’s a stupid atheist. Also, the rest of the post provides no other indications of Christianity.”

  • Alex SL

    As I also wrote in my comment field while submitting my evaluation of the people answering the atheist questions, I felt from the beginning that this was a very dubious affair anyway. A real Turing test would give the judges (whose ability at differentiating the true and pretend believers/unbelievers is being tested) the opportunity to interview the true and pretend believers/unbelievers. Just writing a plausible-sounding paragraph per question is simply too easy if you have any interest in a topic at all, or have heard of Google. Improvising is much harder to pull off.

    The second problem is the self-selection of the candidates. They will all be atheists and Christians who understand the other side fairly well, and not representative for their respective groups.

  • Valhar2000

    You would think then that they must have some successful counter-arguments which they invoke to justify their beliefs. And yet the counter-arguments I encounter are uniformly terrible.

    Maybe they are immune to argument? I mean, maybe they hear a god talking to them, or they “feel Him in their hearts”, or any other one of those weird things you hear people say, and perhaps that sort of thing is more convincing to them than something as mundane as an argument.