Prep for Judging Ideological Turing Test

Thanks for your suggestions, the survey is now LIVE.  

There are fifteen Ideological Turing Test contestants in all, and, starting tomorrow, you’ll get the chance to vote on which of the answers to questions for atheists were truly written by atheists. I’ll be posting all the remaining entries by noon on Friday, and then I’ll open up a link to a Google form for voting. I’m finishing up the survey instrument tonight, and I wanted to open up suggestions for questions.

There are a bunch of entries to reread, so I don’t want to tie you up with a long questionnaire.

The necessary questions:

  • Voter’s religious affiliation*
  • Voter’s guess about authorship of an entry**
  • Voter’s reason for that belief (open-ended, non-mandatory)

*At the most basic, Atheist, Christian, or Other, with a optional specification of sect for Christians and open entry for other. I’ll probably go a little more in depth and add a few questions about whether people were raised in a different faith or have a cultural connection to one they don’t believe.

**Could be a Boolean Atheist/Christian or it could be a Likert scale (probably a four-value scale since I don’t want to leave a ‘undecided’ option or make people think too hard about certainty)

Other possible questions:

I’m inclined to keep it short, but I’d like to hear your suggestions and reasoning in the comments.

About Leah Libresco

Leah Anthony Libresco graduated from Yale in 2011. She works as an Editorial Assistant at The American Conservative by day, and by night writes for Patheos about theology, philosophy, and math at www.patheos.com/blogs/unequallyyoked. She was received into the Catholic Church in November 2012."

  • Josh

    Ask them which they think it is, then ask them optionally how certain they are (out of 10, say). P.S.: You know what would be complex, but cool? Look for words/ideas that appear >5 (say) times across the total corpus of answers, turn them into variables, and run discriminant analysis – see if a statistics can tell these answers apart with any level of certainty. My guess is no, but you'd have a lot to write about if you did find something!

  • KL

    I'm intrigued by the last bullet — "Questions about religious knowledge" — but unsure about what form that would take. What sort of questions/knowledge did you have in mind?

  • http://www.blogger.com/profile/16496144988509668275 Leah

    Probably something a lot like the Pew questions linked, KL.

  • KL

    Right — I apologize for not being clear. Would they be questions about Christian teachings or contain more general world religions content (as the Pew link includes)? And, more to the point, what would your aim be in including them? I like the idea, don't get me wrong — I'm just curious about your goals for the voter questionnaire.

  • http://www.blogger.com/profile/01660960264521298285 The Dude

    What about questions trying to uncover one's toleration of other religions?


CLOSE | X

HIDE | X