Survey says: I have better sex than you

Yesterday I pointed out the curious manner in which journalists wrote up a study showing that non-believers, Catholics and evangelical Christians have smaller brains than Protestants who don’t claim a “born-again” experience. But at least that study, though having a small sample size, was done by real academics at a real university using typical methods of analysis.

Today I see that ABC News is peddling sheer quackery with a big story about a “survey” that was done “on-line.” This is sort of like if Rush Limbaugh asked listeners to call in and give their views on how President Obama is doing and then announced the results of the “survey.” It really doesn’t tell you much.

Anyway, this story is just abominable. The headline tells us “Atheists Have Best Sex Lives, Claims Psychologist.” Which is a pretty weird headline, first off. Like, why do we care what a single psychologist has to say?

Let’s get to the lede:

Darrel Ray, raised a fundamentalist Christian in Topeka, Kan., shed a heavy cloak of guilt surrounding sex after he left the church in the late 1970s, and wondered if his experience reflected that of others.

Today, he has finished research that he said bore out his hypotheses — that religion and good sex don’t mix. In an online survey of 14,500 people who had come from a religious background, he discovered that once they had abandoned their churches, their sex lives improved.

In his survey, “Sex and Secularism,” which he publicized last week, Ray drew a direct correlation between guilt and sexual behavior. Not surprising, but he also learned that guilt eventually subsides.

“We find guilt is a pretty big thing,” said Ray, the author of, “The God Virus: How God Infects Our Lives and Culture.”

OK, then. I wonder if Ray — clearly a subtle man — uses the term “fundamentalist” to mean, you know, “fundamentalist” or to mean “Christian of some kind.” Perhaps we could learn more about the particular upbringing, but I guess that’s asking a lot considering that reporter Susan Donaldson James thinks an unscientific online survey with absolutely no scientific validity deserves the hype for her report.

The story is a complete mess. Take these two lines:

Atheists, he concluded, had the best sex of all. “They can speak with some authority,” he said. “They were raised in very secular homes.”

All his respondents — over 18 and all sexual orientations — had abandoned their churches and described themselves as agnostic or without a religious belief.

So all of his respondents left religious backgrounds but, on the other hand, his atheist respondents were raised in secular homes? One of these two sentences doesn’t work in this story.

Anyway, which academic institution sponsored this “online survey”? Who peer reviewed it? What are Ray’s academic credentials? Let’s see:

Ray, 60, is an independent researcher who has worked as an organizational psychologist in corporations for 30 years. Before that, he spent a decade as a clinical psychologist.

Not just that, but his assistant was a student at University of Kansas. And if that doesn’t impress you, the “online survey” that they conducted “drew 2,500 responses an hour”! That metric could not be more meaningless, could it? I mean, yes, you post a link to your “online survey” over on P.Z. Meyer’s emporium and Metafilter or whatever, and I’m sure you can get 2,500 hits from atheists in a given hour. So what?

Anyway, we’re told that he “only surveyed those who had said they were once religious, not those who today practiced their faith.”

Wait, so how do we know that atheists have “better” sex than non-atheists? What controls were put in place? What was the control group? I mean, if you were to believe in completely uncontrolled online surveys, which you are too smart to do, the most we could say is that atheists who were once religious claim that their sex lives improved once they quit their religion. But we couldn’t say that atheists have better sex than other people because no other people were surveyed. It’s like if I surveyed myself and decided that I have the best sex life in the world.

The story goes on to pretend that there is scientific value in the survey, noting that the “respondents were predominantly highly educated and affluent. They included hetero, same-sex and intersex couples. In the first day, he received 2,500 online responses.”

Wait, so there was one hour that he got 2,500 responses and also the first day total was 2,500? Did Rain Man write this? It’s just bizarre!

Then we get to this weirdness:

“His results make a lot of sense — why people who are religious emphasize guilt in sexual behaviors,” said Tara Collins, president of a multidisciplinary group of researchers at Kansas University’s psychology department, who gave Ray feedback after he presented his survey results.

Collins and others were impressed, but they did express concern about his causal statements and urged him to make some modifications. Ray, she noted, had not looked at the satisfaction level of those who continued to practice their faith.

His research will not be published by an academic institution because it has not been peer-reviewed.

What? What, exactly, were they impressed with? What is there to be impressed with by an online survey of self-selecters? Other than nothing, that is. I had to look Collins up because I couldn’t believe any academic would say anything like this. Turns out she’s not an academic. She’s an undergrad student at KU. Why the story doesn’t describe her as a student, but as “president” of a psych department group is just odd. I’m going with the theory that Susan Donaldson James is also an undergrad student at KU who won a contest to put a story up on a big web site like ABC’s. Please don’t tell me she’s paid to write this stuff.

In any case, did you notice the last line of the excerpt? Which comes in the 20th paragraph, by the way? Of course it’s not peer reviewed! So why did you spend the previous 19 paragraphs pretending it was real scholarship? It’s a freaking online survey. An online survey! Come on, ABC, this isn’t 1997 when the interwebs confused us. It’s 2011 — if you’re a journalist who doesn’t know that online surveys are easily manipulated, hard to control and perhaps completely meaningless, you can turn in your press credentials at the door.

After pointing out that the “online survey” wasn’t peer reviewed, the reporter quotes Mark Regnerus, a Ph.D. in sociology who has written two books on sexual behavior. He points out that the survey was not scientific. We’re into the twenty-something paragraph at this point, keep in mind:

“It appears that it was a ‘fill it out if you want to’ kind of survey that is not random, not nationally representative, and relies entirely on self-selection,” he said. “In other words, they have data from people who felt like filling out a survey on atheism and sex. As a result, I am not surprised at their findings.”

He points to some of his own research on sex and guilt and notes that Ray is not an established researcher at a university and the survey used research methods that don’t meet the standards of most published social science.

Ray responds by pointing out that Alfred Kinsey’s research was done the same way. He’s right. This is the major problem with Kinsey’s results and why he thought that, like 48% of all women were prostitutes or whatever. (Only slightly exaggerating here.) In any case, the story goes back to pretending like Ray’s research is meaningful and points to other discoveries he made.

This story does a disservice to all believers and nonbelievers in that it presents shoddy research as something of value. But it does even greater harm to statisticians and sociologists. Some journalists may not get religion, we know. But they sure as heck don’t get science or statistics either.

Print Friendly

  • http://blog.emergingscholars.org Mike Hickerson

    There are lots of choice quotes, but, as one who reads many sociological studies, this one blew my mind:

    Ray admits the study was self-selective. “I can’t even imagine any research that is not self-selecting,” he said.

    Isn’t selection bias addressed in first-year research methods courses in most graduate programs?

  • http://getreligion.org Bobby

    Did Rain Man write this?

    Rain Man wasn’t this stupid. Definitely not this stupid.

  • http://www.ecben.net Will

    On the other hand, I remember reading something in FIRST THINGS about a “study” where “born-again Christians” reported having more and more satisfying sex than other respondents. Can anyone retrieve that?

  • Suzanne

    I agree that this study is garbage. But I’d like to point out that not all “on-line” studies are inherently worthless. There’s plenty of legitimate, peer-reviewed social/behavioral research that’s conducted online. It’s about how you recruit, how you address bias, etc. Online surveys aren’t appropriate for every survey, but they can be useful in research.

  • Leo

    I’m an atheist, and I agree with you that the methodology on this study was horrible. I think there are many grounds on which to criticize organized religions and certain religious dogma, but it’s really embarrassing to see my fellow atheists embracing bad science to score political points.

  • http://www.getreligion.org Mollie

    Suzanne,
    This is true. You could, for instance, email a nice statistical sample a link to a survey with an authorization code, for instance. But this was not such a survey.

  • Dave

    And furthermore, how do we know that “improved” means the same thing to different people? What’s the metric for “better” sex? Frequency of contacts? Intensity of experience? Variety of… well, I trust you get the point.

  • Ben

    Some journalists may not get religion, we know. But they sure as heck don’t get science or statistics either.

    I’m guessing the author knows a lot about a certain type of statistics: web traffic. :-)

    Nice writeup, Mollie.

  • http://platytera.blogspot.com/ Christian

    Thinking about my own life, sex with religion is way, way better than sex without religion.

  • http://www.ecben.net Will

    We also have to speculate on how much this was “filtered” by the reporter. Jumping from “self-selective” to “self-selecting” in successive sentences does not inspire confidence in her accuracy, let alone grasp of the material.

  • Jon in the Nati

    Bad study, bad science, bad reporting of the results.

    But if we’re going to be serious about the results, such as they are, then we should at least be honest about what we’re talking about. The researcher says the gist of the survey is that ‘religion and good sex don’t mix’. He then goes on to talk at length about guilt over sins, etc; so what he is really saying is that religion and illicit sex don’t mix. I have a hard time believing that most well adjusted people feel terribly guilty about the sex they have with their spouses.

    Frankly, the idea that religion and illicit sex do not mix should go without saying. That atheists and others have a problem with this is their problem, not the religion’s problem or the church’s problem.

  • http://www.ecben.net Will

    But Everybody Knows that Religion (which is monolithic and homogeneous) regards Sex as evil, period. Are you saying that Everybody could be WRONG?

    Meanwhile, I noticed something else… the researcher, or the reporter, or the headline writers, seems to equate “leaving one’s religion” with “atheist”. Obviously not so. A Buddhist who is converted to Christianity, or vice versa, has left his religion.

    This parallels Heinlein’s statement pretending to define “cult” as “that which one joins as an adult”, instead of being “born into”…. which makes EVERY religion a “cult”.

  • Jerry

    A side note about online surveys. Nate Silver who now writes a blog including analyzing political poll results for the New York Times has looked at self-selected online polls. He’s found that it is possible, with careful methodology, to find meaningful results. The key, of course is careful methodology.

    For every online polling firm like YouGov, which thinks carefully about its polling and has had some encouraging results, there are others like Zogby Interactive that make no real effort to ensure that they have drawn anything resembling a scientific sample. …

    http://fivethirtyeight.blogs.nytimes.com/2010/10/04/the-uncanny-accuracy-of-polling-averages-part-iv-are-the-polls-getting-worse/

    So I don’t think that online surveys are a priori automatically badly done or wrong, but that it’s hard to do it right.

    And I also think that if Dante were writing today about what some call “news”, there would be a level reserved for those that manage to get both religion and science wrong in a single story.

  • Chris

    My favorite line from the ABC news piece was:
    “His research will not be published by an academic institution because it has not been peer-reviewed.”
    Ah–so there is a difference between “publicize” and “publish”! Publicizing is when you do an on-line survey, and get an interview with ABC. Publishing is when you submit the methods and the results to other experts in the field, and they “peer review” it. Unfortunately, I suspect this data will never be published–only publicized!

  • Martha

    It seems to me that what Mr. Ray is saying with this “survey” is that “better” means “more”.

    In other words, once these people left their churches or gave up their faith, they starting having more sex; instead of waiting until marriage, they began co-habiting or having sex with people they dated.

    Does that mean better sex, though? I mean, if you’re a former Baptist and then you switch to a denomination that allows you to drink beer, does that mean you have a “better” drinking life than Baptists who don’t imbibe?

    By this metric, being an alcoholic would be the ‘best’ outcome of all!

    No, I don’t agree that promiscuity is better sex, as you can probably tell :-)

  • Martha

    Jerry, I think Dante already has this covered. Eighth Circle of Hell, the Ten Malebolge which house the fraudulent. Second Malebolge, those who were fraudulent by word – this is the flatterers, but I think liars, schemers, false advertisers, and those who mangle stories or slant them would also be included here: submerged in excrement.
    :-)

  • http://www.ecben.net Will

    It looks like the “metric” was people’s answers to “Is your sex life a)better b)worse c)the same).”

  • Dave

    I just had a stunning insight. It gets better when you have a larger hippocampus.

  • http://blog.emergingscholars.org Mike Hickerson

    Mollie,
    One correction to your post. It looks like this page is a list of graduate students in the KU social psychology doctoral/masters program. So Tara Collins is a 4th year graduate student, not an undergrad. That’s a bit better, though I’m not sure why she couldn’t have been identified as a doctoral student.

  • http://www.getreligion.org Mollie

    Mike,

    That’s definitely better, although it’s still kind of odd to quote a student without identifying her as such.

  • Jerry

    Martha, thanks for the pointer to the proper circle of hell.

  • http://ingles.homeunix.net/ Ray Ingles

    michael –

    So this is what American atheism in the 21st century has come down to?

    Because what any one atheist does, all atheists must perforce be guilty of?

  • Martha

    Jerry, I think it’s important we should all know our ultimate destinations clearly.

    (And yes, I have an uncomfortable feeling about which circle I’m bound for if I don’t pull my socks up sharpish).

  • http://jotsandtittles.wordpress.com/ Raj Rao

    I’m sure that GetReligion has already mentioned this but…

    Have you folks considered that some “articles” at very prominent news sites are deliberate fakes, engineered to:

    (1) increase traffic to the websites

    and

    (2) maintain that increase by getting people to register an account at the site.

    So you write something that totally gets people angry and so much so that they will immediately want to respond. However in order for them to respond, they have to register an account. And what does that ensure? It ensures that they will keep coming back again and again to their site rather than some other site. It also gets people on the email list which is good for advertising.

    This is my growing suspicion. Why? Well. Because I find that some prominent sites will publish an article in which they will give information X, Y, and Z on a topic.

    Later on they will publish another article on the same topic and not neglect to take into account Y and Z. Go figure.

    My point is that the atheism & sex article might be just such a faked article.

    Ciao 4 Now,
    ~ Raj Rao

    P.S. I think that there is possibly a third reason for quack articles. As we all know, newspapers, magazines, etc., are dying out. However they hold greater weight than soft media in terms of research, accountability and such. If you are writing a research paper, you can trust them far more than a soft copy which is here today and gone tomorrow. Quack articles push us back to depending on hard media rather than soft.

  • http://www.ephesians4-15.blogspot.com/ Randy

    The story does actually tell you something about what atheists think about sex. That is that sex is better without religion. I don’t think it is true. But it is important. Like people who commit adultery think sex is better outside marriage. It does not mean good sex is impossible in marriage. More likely that good sex prevents adultery and atheism. People don’t blame themselves for lousy sex lives. They blame the rules.

  • Dave

    I see no a priori connection between theism/atheism and attitude toward sex. It’s perfectly possible for the theist to believe that sex is a divine gift, enjoyed at its best when embraced consistently with divine guidance; and for the atheist to belive that religion hampers sex with repressive rules and it’s best enjoyed unencumbered by such rules.

  • http://socraticgadfly.blogspot.com SocraticGadfly

    Lying for the sake of New Atheism: http://socraticgadfly.blogspot.com/2011/05/lying-for-sake-of-new-atheism.html

    That’s why, more and more, I’m liking the old phrase “secular humanist” as a self-descriptor.