Parsing that “98% of Catholics use contraception” figure

I’ll be back with a longer post on the actual substance of the contraceptive mandate and my ideas about the reasonable limits to the idea that religious mandates exempt you from the law, but I can’t pass up the chance to geek out about social science statistics and methodology.

You’ve all probably heard the “98% of Catholic women use contraception” statistic cited by now, and people have started to take a closer look at the relevant study.  Let me chat you through some of the objections.

Who was polled?

The survey done by Guttmacher didn’t include all women.  Only women who met these three criteria were included in the computation

1. Between the ages of 15-44
2. Not pregnant, post-partum, or trying to get pregnant at the time of the survey
3. Sexually active (had sex at some point in the last three months)

This seems like a pretty good sampling frame to me.  It looks like the researcher was trying to see what percent of women who were having recreational sex were using contraception.

The most intuitively correct restriction is age.  We don’t expect children to be using birth control, so including them in the survey just means you’re goosing the ‘no’ responses.  That would be reason enough to exclude them, since you’re not trying to learn things about their demographic, but there’s a more important methodological reason to keep them out of your sample.

If both populations (Catholic and non-Catholic) had the same age distribution, including children would just drive down the percentages for both subsets.  But if the two groups were demographically mismatched, whichever group was younger relative to the other subset would look like it had a lower contraceptive rate.

But…

This means the study doesn’t have anything to say about all Catholic women.  It simply says that Catholic women who are having non-deliberately-procreative-sex use contraception at about the same rates as any other women in those circumstances.

So, it would be good to report what percentage of Catholic women aged 15-44 fall into that subset and contrast that proportion with other religious groups.  And these two numbers should be reported together.

The table above doesn’t do it, since it just reports the percent of women who have ever been sexually active outside of marriage.  As far as I can tell, the number I’m looking for wasn’t calculated by the researchers.

What were they asked?

The first problem may just be sloppy reporting (and non-ideal, but passable survey design).  The second one is more serious.  Take a look at the graph from the study below:

The 98% number is the result of only counting the NFPers as ‘non-contracepting.’  The trouble is that it’s not clear exactly what “other” and “no method” refer to.  According to the methodology:

The category of “other” methods mainly consists of withdrawal but also includes less common methods, such as suppositories, sponges and foams.

I think it may vary by religion whether ‘withdrawal’ really counts as contraception.  (Someone can fill me in with a comment).

The big problem is that there’s no explanation of what “no method” means in the paper.  Get Religion assumed it meant that the women were not using contraception, which certainly seems plausible.  However, it may also mean that women answered yes to “Are you currently using some kind of contraception” but refused to specify the type in the followup question.  I’ve seen this style of notation for academic studies before.  A reporter should have called up and asked for clarification.

So, the percent using contraception is at most 98%, and if you were really being conservative, you should probably cite 83% (condoms, IUDs, the Pill, and sterilization).

So, in a nutshell…

Catholic women who are in a position where they’d be tempted to contracept, use contraception at about the same rate as everyone else in those circumstances, and that rate is somewhere between 83-98%.  How many Catholic women are in this situation in the first place?  Beats the heck out of me!