Turing Test Update

In light of the interest in the ‘Atheist’ round of my Ideological Turing Test, I’m extending the deadline for submitting answers until midnight on Tuesday night, so keep forwarding the link around!

The answers to the ‘Christian’ slate of questions will start going up early Monday morning. There are fifteen answers in all (from the same slate of contestants who participated in the first round). If you don’t want to have too much to slog through when the ballot goes up on Friday, I recommend keeping a note of your guesses as you read the answers during the week.

Sign up for next time

People have been using the comments to ask if they can participate in another iteration of this test but trying to keep tabs of requests all over the site is a recipe for disaster. If you’d like to be considered to participate in another run of the Turing Test, fill out this form, so I have your email and biographical information.

I’d never run this again with so many participants at once, but I’m open to running smaller scale contests every now and again. We can revamp questions and maybe even set up a real-time contest in the future. Suggestions welcome.

If you want to know when future contests/ballots/results are being posted, you can follow this blog by RSS or by email (sign up in the sidebar).

Survey Format

I know a number of you are frustrated by the format of the survey, and, since I’ll be releasing another one for the same purpose on Friday, I’m very open to suggestions. Here’s what I need: online survey software that

• is available for free (I don’t make money off this blog)
• gives me access to the raw data, not just summaries (looking at you, SurveyMonkey, grump)
• lets me export the data in some reasonable format (.csv will do)
• makes it easy to set up open-ended questions as well as the table of ratings

Or, absent that, I need someone to explain how I could persuade Google Forms to render those bit.ly links as hyperlinks (just typing html into the question did bupkis). Leave suggestions in the comments!

I’d also take suggestions for better labels for the four value Likert scale than “Christian/Lean Christian/Lean Atheist/Atheist.” I didn’t want to use words like “strong/weak” that could be interpreted to refer to the contestant’s beliefs on a liberal/conservative spectrum.

Methodological Rigor?

A number of commenters have complained that this test is not particularly rigorous. That’s absolutely true, and it’s fine by me. If I were trying to make a definitive claim about whether atheists or Christians in the general population were, on average, more likely to be able to parrot each other’s religious beliefs, I’d have to be a lot more careful about how I ran the test, and I wouldn’t be accepting votes from everyone who stops by on the internet (at least, not without a lot more demographic questions in the ballot so I could try to jury-rig a way to weight responses appropriately.

I’m not out to prove something, I just want to start a discussion. The reasons the ~1000 of you who have filled out the survey so far have given for your guesses are of as much interest to me as the answers the atheists and Christians wrote. Stick around for the results in two weeks, after I run the atheists-pretending-to-be-Christians round, and I promise food for thought.

And if anyone wants to run this kind of experiment with a rigorous methodology, let me know, and I’ll link you and try to send you respondents (unless that messes up your sampling frame).

P.S. This means if you link to my results and claim they prove your side is better informed and thus correct, I will bop you on the nose.