Choosing Questions for the Turing Test Judges

Before the end of July, you the readers will start reading and evaluating the contestants in this year’s Ideological Turing Test: Sex and Death Edition.  Last year, I added a few questions to the ballots, but I didn’t make much use of them, so I’m inclined to drop the extras and save you time (since you’ll be filling out the questionnaires once for each entry).

Here’s last year’s sample ballot.

And here are the questions I plan to use this year:

  1. What is your best guess of which side this contestant is REALLY on?
    • Very Likely Christian
    • Likely Christian
    • Likely Atheist
    • Very Likely Atheist
  2. Which is the best description of your religious beliefs?
    • Christian
    • Atheist
    • Other: _____
  3. Have you ever changed religious beliefs?
    • Yes, changed belief about God
    • Yes, changed big umbrellas (e.g. Islam -> Judaism)
    • Yes, changed sects within one tradition (e.g. Baptist -> Lutheran)
    • No
  4. Whether or not you agree with the entry, how intellectually respectable did you find it?
    • Not at all, I may even suspect if of being a strawman
    • It’s reasonably coherent, but I wouldn’t expect it to change minds
    • Quite a lot, I’d like to have a coffee discussion/argument with this person


Edits?  Potential ambiguities?  Need for rewordings?  Pleas to restore or strike questions?

"Well, I would love to know if you now believe that homosexuality is intrinsically disordered."

Go Ahead, Tell Me What’s Wrong ..."
"Any chance of you ever addressing the evidence that led you to accept the truth ..."

Letting Go of the Goal of ..."
""Wow, an unevidenced assertion from a religious dipshite. "Your quotes are the evidence and reason ..."

This is my last post for ..."
""Congrats on leaving your brain behind!"Comments like yours are why lots of atheists leave atheism. ..."

This is my last post for ..."

Browse Our Archives

Follow Us!

What Are Your Thoughts?leave a comment
  • Roki

    I think in question three, you mean “e.g.” rather than “i.e.”

    And since that’s the biggest critique I have of the questions, I say, well done. I’m curious (as always) to see how the ITT turns out.

    • LeahLibresco

      Ugh, I mess this up all the time. (But, until this year, I wasn’t even aware my instinct was wrong, so: progress!)

      • Roki

        The only way I keep them straight is to English e.g. into “example given” – which is not too rough a translation of the Latin.

        • kristen inDallas

          and i.e. can be roughly associated with “in essence”

          • Roki

            Oh, that’s good. But it does stir up the whole esse/essentia debate again…. 😉

  • Brutus

    5: How much do you agree with this entry?

    • Roki

      Perhaps if followed by, “How does your agreement/disagreement affect your judgment on the authenticity of the entry?” But then we’re sliding very much toward evaluating the judge, rather than evaluating the entry.

      • Brutus

        Authenticity? What property is that?

        • Roki

          The property of whether the entry appears to be a statement by an actual adherent of that position. In other words, whether your agreement/disagreement with the statement affects whether the statement can fool you or not.

          • Brutus

            So, whether you think that you are biased to think that arguments for positions you disagree with come from religions that you are or are not?

            My intention was to measure if people are better or worse when they agree with the position, which is done by statistics on the ‘agree’ question and the ‘accurate’ statistic.

            Perhaps also a question “How well-formed is this argument?” (Hypothesis: People will correlate poorly-formed arguments with false-flag arguments. On many arguments, the ‘well-formed’ score from people who believe it to be false-flag will be lower than the score given by people who think it genuine, possibly excepting arguments that are consistently judged as horrible or great.)

          • Roki

            Ah, I see where you’re coming from now.

            An interesting question, but still, it’s more to evaluate the judges than to evaluate the entries.

  • LeahLibresco

    Yup, but the entrants are writing for those two perspectives, so that’s what I need to match to to see if they convinced their own community

  • TheodoreSeeber

    Looks good to me. Since I’m not a contestant, I’ll be voting. My only question is on survey question #3- do you include reverts in “big changes”, given that I seem to have traveled in a huge circle both religiously and politically?

    • TheodoreSeeber

      In that, depending on how you look at it, I could fit A, B, or D but never seriously C. “Ever” seems to cover reverts.

      • Mariana Baca

        It can’t be both D and and A or B. If it is A or B, it can’t be “not ever.”

        A is a broader category, so pick A.

    • LeahLibresco

      Yes, reverts count

  • “I may even suspect it* of being a strawman,” I presume. The “it” is a little weird; I would sugget “them,” maybe? Or a fancy gender-neutral pronoun.

    • Martha O’Keeffe

      Leah is speaking of the entry, not the entrant, so the “it” refers to “the piece that you have just read”.
      Although I may be behind the times; does progressive thought in America now extend to giving inanimate objects the courtesy of choice of gender preferences? 😉

      • I guess then what’s weird is refering to an entry as a strawman? I tend to think of strawmen as being fictitious persons, not fictitious entries. Am I getting the idiom wrong?

        • InkDuBlog

          As far as I know, strawmen are fictitious arguments. (Also, Martha, progressive thought gave inanimate objects a neuter gender; almost any other language has lots of gendered inanimate objects, like feminine tables or masculine forks. Sometimes it’s fun to drag them into English.)

          • Mark Twain’s The Awful German Language is hilarious and partially based on that joke.

            (But it’s not entirely foreign to English either, countries and ships can often be called shes).

          • I had thought that the idiom “strawman argument” meant a fictitious argument, so I assumed that “strawman,” on its own, would mean the person who espoused a fictitious argument. But maybe “strawman” is being used as shorthand for “strawman argument.” I could believe that.

  • Martha O’Keeffe

    Looks good and I anticipate being completely wrong in all of my decisions when I come to vote (the one person I am going to pick as plainly, obviously, blatantly an atheist/theist who didn’t even try to construct a convincing theist/atheist argument is going to turn out to be the other way).
    I love these kinds of things! 🙂

  • EMMilco

    “I may even suspect IF of being a strawman.”

  • Kristen inDallas

    On question 3 – I wouldn’t really be sure what the intended differences are between “Yes, changed belief about God” and “Yes, changed big umbrellas.” On a strict interpretation, I’m not sure I’d fit into either. But loosely, I might qualify as both. Is Confused and uncertain about stuff for a time an umbrella? (I used the term agnostic but it wasn’t really correct) And when you say changing belief about God, do you mean specifically the fact of belief (yes, no or maybe) or is believing about Him in a different way, or changing the qualities attributed to him, also counted as a change?

    • Mary E.

      Perhaps “Yes, changed beliefs about the existence of God?” (Assuming that existence or non-existence is the gist of the question.) Or, “Yes, changed beliefs about the role of God.”

  • “I wouldn’t expect it to change minds” isn’t a great way to express middling intellectual respectability. People are stubborn. I might want to have coffee with someone even though I don’t think their entry clears the very high “mind changing” bar.

    • LeahLibresco

      Alternate suggestion?