You may have seen the New York Times‘s much shared infographic on contraception failure rates, and I’m here — statistics cop badge in hand — to tell you that it ain’t necessarily so. There are some huge methodological flaws in the way that the NYT chose to model the risk of unplanned pregnancy over ten years, and I’m explaining just how badly they went wrong at The American Conservative.
The NYT generated these graphs by looking only at failure rates for contraception over one year, and then doing some arithmetic to model the chance of failure over the long term… It’s the right kind of statistics to use if you’re talking about some fixed risk, like the chance of winning a coin flip. But the chance of unintended pregnancy isn’t likely to remain constant over a 10-year period.
For one thing, the women who get pregnant in their first year of using a particular form of contraception may be different than the rest of the women in their cohort. A significant proportion of the women who wind up pregnant may be more fertile than average (or are having sex with a guy with excellent sperm). Irregular cycles may make it harder for them to use NFP charting successfully. They may work schedules that make it much harder to stick to an “every day, same time” form of contraception like the Pill. The may be college students having a lot of tipsy sex, and not remembering to manage condoms very well.
Once those women get pregnant in the first year, they’re out of the analysis, as far as the NYT is concerned. But the charts continue to use the risk probabilities set by the most unsuccessful group of contraceptors as the baseline risk for everyone who remains.
There aren’t very good numbers on long term risk of pregnancy for contracepting couples (for reasons I discuss in the article) but the NYT‘s bad model is worse than just throwing up our hands and saying that the problem is too hard to approximate well.
Particularly when you take this approach out of the abstract, hypothetical domain of a journal and put it in the paper of record, where people are likely to act on it.