The Ideological Turing Test starts this week, and I’ve put together a generic Google Form so you guys can give feedback on the survey instrument. You can test drive it here. Make sure you give me feedback in the comments of this post, not on the form.
I’d definitely like to know if you have any technical problems, and this is your last chance to give feedback on the questions. I’m using a four item Likert scale for judging, since I’m guessing that upping the number of choices will not help people drill down. I’ve been sticking with the “Very Likely Christian” “Likely Christian” “Likely Atheist” and “Very Likely Atheist” labels. Do people still like these?
I’m adding a question about how compelling people find the entries. Last year, some Christians deliberately tried to imitate “boring” atheists. That’s fair tactics, but I would have liked to know if the atheist voters who passed them agreed that the answers were dull. So this year, I want to add a question about how compelling/attractive the entries are, regardless of whether you accept their premises. I’m not sure of the best way to go about it, though.
Currently, I’ve got category labels, but I’m not very happy with them. Perhaps it would be better as just “Please rate how compelling you found this entry on a scale of 1 to 5”? I’m concerned atheists and Christians may bring radically different ideas of what ‘compelling’ denotes to the survey. I could break it up into two questions, one on how internally consistent and logical you found the entry and one of the attractiveness of the worldview being described, regardless of it’s truth value. Advice?
Make sure you let me know if there are other demographic questions you want included. Keep in mind that, since I’m doing one form for each entry, I don’t want the non-essential questions to be too long, since you’ll fill them out each time.