Returning to Turing Test Methodology<\/title>\n<meta name=\"description\" content=\"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Returning to Turing Test Methodology\" \/>\n<meta property=\"og:description\" content=\"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html\" \/>\n<meta property=\"og:site_name\" content=\"Unequally Yoked\" \/>\n<meta property=\"article:published_time\" content=\"2012-06-26T16:49:04+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2013-01-22T01:49:46+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/wp.patheos.com.s3.amazonaws.com\/blogs\/unequallyyoked\/files\/2012\/06\/ITT-sample-size-1024x739.png\" \/>\n<meta name=\"author\" content=\"Leah Libresco\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Leah Libresco\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html\",\"url\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html\",\"name\":\"Returning to Turing Test Methodology\",\"isPartOf\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#website\"},\"datePublished\":\"2012-06-26T16:49:04+00:00\",\"dateModified\":\"2013-01-22T01:49:46+00:00\",\"author\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/17ee17592b35b40040d5f5f7ea5ab464\"},\"description\":\"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the\",\"breadcrumb\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Returning to Turing Test Methodology\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#website\",\"url\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/\",\"name\":\"Unequally Yoked\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/17ee17592b35b40040d5f5f7ea5ab464\",\"name\":\"Leah Libresco\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/20e1e6a3a94c4e7928687804a41d888d?s=96&d=mm&r=r\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/20e1e6a3a94c4e7928687804a41d888d?s=96&d=mm&r=r\",\"caption\":\"Leah Libresco\"},\"description\":\"Leah is the author of Arriving at Amen and Building the Benedict Option. Her further writing can be found at leahlibresco.com.\",\"url\":\"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/author\/leahlibresco\"}]}<\/script>\n","yoast_head_json":{"title":"Returning to Turing Test Methodology","description":"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html","og_locale":"en_US","og_type":"article","og_title":"Returning to Turing Test Methodology","og_description":"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the","og_url":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html","og_site_name":"Unequally Yoked","article_published_time":"2012-06-26T16:49:04+00:00","article_modified_time":"2013-01-22T01:49:46+00:00","og_image":[{"url":"http:\/\/wp.patheos.com.s3.amazonaws.com\/blogs\/unequallyyoked\/files\/2012\/06\/ITT-sample-size-1024x739.png"}],"author":"Leah Libresco","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Leah Libresco","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html","url":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html","name":"Returning to Turing Test Methodology","isPartOf":{"@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#website"},"datePublished":"2012-06-26T16:49:04+00:00","dateModified":"2013-01-22T01:49:46+00:00","author":{"@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/17ee17592b35b40040d5f5f7ea5ab464"},"description":"I'm a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian and Atheist rounds of the","breadcrumb":{"@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.patheos.com\/blogs\/unequallyyoked"},{"@type":"ListItem","position":2,"name":"Returning to Turing Test Methodology"}]},{"@type":"WebSite","@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#website","url":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/","name":"Unequally Yoked","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/17ee17592b35b40040d5f5f7ea5ab464","name":"Leah Libresco","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/20e1e6a3a94c4e7928687804a41d888d?s=96&d=mm&r=r","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/20e1e6a3a94c4e7928687804a41d888d?s=96&d=mm&r=r","caption":"Leah Libresco"},"description":"Leah is the author of Arriving at Amen and Building the Benedict Option. Her further writing can be found at leahlibresco.com.","url":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/author\/leahlibresco"}]}},"_links":{"self":[{"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/posts\/3721","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/users\/127"}],"replies":[{"embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/comments?post=3721"}],"version-history":[{"count":0,"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/posts\/3721\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/media?parent=3721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/categories?post=3721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/wp-json\/wp\/v2\/tags?post=3721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

{"id":3721,"date":"2012-06-26T12:49:04","date_gmt":"2012-06-26T16:49:04","guid":{"rendered":"http:\/\/www.patheos.com\/blogs\/unequallyyoked\/?p=3721"},"modified":"2013-01-21T20:49:46","modified_gmt":"2013-01-22T01:49:46","slug":"returning-to-turing-test-methodology","status":"publish","type":"post","link":"https:\/\/www.patheos.com\/blogs\/unequallyyoked\/2012\/06\/returning-to-turing-test-methodology.html","title":{"rendered":"Returning to Turing Test Methodology"},"content":{"rendered":"\n<\/head>

I\u2019m a little disappointed in the commentariat for not bitching me out when I showed you the winners and losers in the Christian<\/a> and Atheist<\/a> rounds of the Turing Test<\/a> without ponying up sample size numbers. \u00a0This year, each entry had its own survey link, so the number of respondents was not constant across a round. \u00a0A fairly predictable trend emerged.<\/p>\n

$\"\"$ <\/a><\/p>\n

Note:<\/em><\/strong> This isn\u2019t the total number of respondents, it\u2019s the N for the number of self-declared atheists judging the first round and self-declared Christians judging the Christian round. \u00a0Looking at how well people do at spotting fakes in the group they don\u2019t<\/em> belong to is interesting, but I exclude those people from the figures that identify the winners.<\/p>\n

Unsurprisingly, there\u2019s a big drop off as people got tired of reading and judging entries. \u00a0The difference between participation in the Christian and Atheist round is mainly a function of how many bloggers in each category I got to promote the test to their readers. \u00a0I\u2019ll probably do this again next year, and I\u2019m wondering if readers have any advice on tweaking the methodology to avoid this\u00a0precipitous\u00a0drop in response rates.<\/p>\n

Making people answer all the questions at once is a pretty big\u00a0commitment, so it depresses response rates across the board. \u00a0I got lucky last year, in that Andrew Sullivan linked the test, so my click throughs were high enough to be able to bleed off a lot of people and still have good numbers. \u00a0I can\u2019t count on huge links like that when I design my methodology. \u00a0 \u00a0The big upside of the all-in-one survey is that I can compute each participant\u2019s accuracy and then maybe compare whether converts and deconverts did better than people who have stuck to only one philosophy.<\/p>\n

Ideally, I\u2019d be able to randomize the order that people answered in, but that seems pretty impractical. \u00a0And that set-up would make it awfully hard to read a couple, vote, and then take a break before you got burned out. \u00a0Anything that would let you store answers as you go, and then return to them later would be beyond my webdesign\/database skills at present. \u00a0(For last year\u2019s all-in-one survey, I recommended people keep a paper and pen by their computer).<\/p>\n

Another way to track responses across separate surveys would be to get people to generate userids that wouldn\u2019t give too much data away. \u00a0(Think last four digits of phone number followed my MMDD birthday). \u00a0Something easy for you to remember and enter on each individual survey, not too likely to be duplicated, and not that dangerous to share with me.<\/p>\n

I\u2019d be interested in your thoughts on the tradeoffs of these approaches and advice on tweaks. \u00a0Mind you, I\u2019d really love for next year to be the round we try out chat logs instead of essays, so a more radical overhaul may be necessary.<\/p>\n<\/body><\/html>\n","protected":false},"excerpt":{"rendered":"