Turing Test Update

Turing Test Update July 10, 2011

In light of the interest in the ‘Atheist’ round of my Ideological Turing Test, I’m extending the deadline for submitting answers until midnight on Tuesday night, so keep forwarding the link around!

The answers to the ‘Christian’ slate of questions will start going up early Monday morning. There are fifteen answers in all (from the same slate of contestants who participated in the first round). If you don’t want to have too much to slog through when the ballot goes up on Friday, I recommend keeping a note of your guesses as you read the answers during the week.

Sign up for next time

People have been using the comments to ask if they can participate in another iteration of this test but trying to keep tabs of requests all over the site is a recipe for disaster. If you’d like to be considered to participate in another run of the Turing Test, fill out this form, so I have your email and biographical information.

I’d never run this again with so many participants at once, but I’m open to running smaller scale contests every now and again. We can revamp questions and maybe even set up a real-time contest in the future. Suggestions welcome.

If you want to know when future contests/ballots/results are being posted, you can follow this blog by RSS or by email (sign up in the sidebar).

Survey Format

I know a number of you are frustrated by the format of the survey, and, since I’ll be releasing another one for the same purpose on Friday, I’m very open to suggestions. Here’s what I need: online survey software that

  • is available for free (I don’t make money off this blog)
  • gives me access to the raw data, not just summaries (looking at you, SurveyMonkey, grump)
  • lets me export the data in some reasonable format (.csv will do)
  • makes it easy to set up open-ended questions as well as the table of ratings

Or, absent that, I need someone to explain how I could persuade Google Forms to render those bit.ly links as hyperlinks (just typing html into the question did bupkis). Leave suggestions in the comments!

I’d also take suggestions for better labels for the four value Likert scale than “Christian/Lean Christian/Lean Atheist/Atheist.” I didn’t want to use words like “strong/weak” that could be interpreted to refer to the contestant’s beliefs on a liberal/conservative spectrum.

Methodological Rigor?

A number of commenters have complained that this test is not particularly rigorous. That’s absolutely true, and it’s fine by me. If I were trying to make a definitive claim about whether atheists or Christians in the general population were, on average, more likely to be able to parrot each other’s religious beliefs, I’d have to be a lot more careful about how I ran the test, and I wouldn’t be accepting votes from everyone who stops by on the internet (at least, not without a lot more demographic questions in the ballot so I could try to jury-rig a way to weight responses appropriately.

I’m not out to prove something, I just want to start a discussion. The reasons the ~1000 of you who have filled out the survey so far have given for your guesses are of as much interest to me as the answers the atheists and Christians wrote. Stick around for the results in two weeks, after I run the atheists-pretending-to-be-Christians round, and I promise food for thought.

And if anyone wants to run this kind of experiment with a rigorous methodology, let me know, and I’ll link you and try to send you respondents (unless that messes up your sampling frame).

P.S. This means if you link to my results and claim they prove your side is better informed and thus correct, I will bop you on the nose.

"I'd love to see a video of how it works. keranique shampoo reviews"

Welcome Camels with Hammers to Patheos!
"Logismoi (the plural of logismos) are a fairly simple concept; they are whispers from either ..."

Logismoi, Vampires, and Other Intrusive Thoughts
"I imagine I’ll do a lot more reading and pick a lot more fights over ..."

A little about the queer stuff
"You are part of a search and rescue for lost Catholics.Regular updates to the countdown ..."

I’m keynoting at a Con for ..."

Browse Our Archives

Follow Us!

What Are Your Thoughts?leave a comment
  • When I first read the labels I thought that "lean" meant "weak" ideologically. Perhaps it would be more clear if they were "Definitely Atheist" "Probably Atheist" etc.? Perhaps on a next installment, a "I honestly can't tell" choice might also be in order.

  • Just wanted to clarify – when you say "from the same slate of contestants who participated in the first round" I assume they aren't necessarily in the same order?

  • Correct. I'm just using the random number generator in Excel to determine order each time. P.S. Please no one complain that this isn't a perfect random number generator. As I said above, I'm not going for perfect rigor.

  • Anonymous

    Someone who complained about experiment-affecting bias in Excel's RNG for 15 integers would really just be displaying their own ignorance anyway…

  • Sarah

    I think having 15 participants is nice–fewer than 10 would be less fun. Maybe you could convince some other blog to host another round, if it's too much work to repeat? I would be especially interested in seeing a contest based on another religion or on multiple religions (the division could be atheist-theist, and the atheists could choose which religion to fake). The current version is fun, but it reinforces the thing that always makes me a little uncomfortable about your blog: the implicit atheism/Christianity binary, where both the atheism and the Christianity tend to be of a particular and narrow kind, and where other belief systems aren't explicitly dismissed, but are somehow not worth considering or knowing about. I know this blog was set up in a particular atheist/Catholic context, but it wouldn't hurt to remember the rest of the world now and then.

  • Anonymous

    I'm with Sarah on also wanting to see a more general atheist-vs-theist set!