Death of Religion Predicted by Demographers Torturing Statistics

Because what’s the use of a blog that brings together Christians and atheists if it doesn’t take time to snark on bad science reporting/bad study methodology?  I like geeking out about this, but if it’s not your cup of tea, never fear: the next post in queue covers the ex-gay lecture I mentioned I was attending on Friday.

UPDATE: Thanks to Darksmiles for pointing out an error in this post.  I misread the scale of some of the graphs in this paper, since the study listed them as “percentages” in the legends and the actual graphs axes were labelled as proportion.  That makes a hundredfold difference and completely nixes one of my critiques of the paper.   I’ve dropped that bit, but left up and modified the parts about extrapolation and chart making.

The blogosphere has been buzzing about a new study that predicts the coming extinction of religion.  Some of the  hype is, as always, the fault of the BBC article most people are working from (the BBC is notoriously terrible at science writing), but I’ve had a look at the actual article on arXiv, and I think the researchers are overstating their results, or, at the very least, framing them poorly.

To briefly recap the methodology, the researchers model future religious affiliation as being primarily dependent on the attractiveness of a particular religion.  They define attractiveness to be primarily a function of  how many people are currently affiliated with the religion, since a community with few parishioners will not benefit from the same level of pride in unity, resources, energy, etc as would a more robust parish.

If you have a little mathematical intuition, you won’t be surprised to learn that this model predicted a decline in religious affiliation that would grow faster and faster until religion was extinct.

Ok.  So let’s have a quick chat about extrapolation and modeling.  Take a look at these plots from the paper:

So, the time scales for the data they’re considering extend til 2005 in the Aland islands of Finland, 2000 in one canton of  Switzerland and a province of Vienna, and 2050 in the Netherlands?!?

There’s no excuse for that graph design.  The researchers, who are ostensibly trying to accurately approximate past trends, have no reason to think that trend will necessarily hold 50 years in the future.  The rescaling of the graph makes it hard to see the actual data.  When I recrop their graph to purge the no-data section, a fair amount of the projected curve plotted is totally irrelevant.

There was never a reason to have that part of the graph included in the figure, since the researchers don’t have a strong reason to assume their projection is correct.  This study was entirely focused on checking whether a model was a good fit for pre-existing data; they don’t have a way to check how well it will do at predicting future trends.

That’s not to say it might not predict future trends, only that we can’t know that yet.  As years pass, we can check whether new demographic information matches this model’s predictions, but, if they didn’t, all the researchers would have is a decent way to summarize known data.

It’s also worth remembering that extrapolation can go backwards as well.  If we ran the equations further into the past, non-religious affiliation would approach zero.  What’s interesting to talk about is where the rise begins and what prompts it.  The data as they stand could be of interest to religious historians who would try to figure out what variations by geographical area would have jumpstarted this process of deconversion at different points in time.

A variety of factors would have influenced when this process began and its initial speed.  Variables and influences just as complex will impact the future trends, and there’s plenty of reason to believe that the researchers’ model of utility associated with popularity will not persist.

To illustrate this point, I want to look at a complaint some religious blogs made about this study.   They claimed the methodology was rigged because the entire model was adapted from previous work the academics did to model the way that declining numbers of fluent speakers of a dialect accelerated the death of that dialect.  How could this model do anything but predict the death of an organization, they asked?

Well, it turns out that sometimes, when a language or dialect is disappearing, the trend doesn’t just continue unabated.  Here’s an interesting example from a study tracking the Martha’s Vinyard accent:

On Martha’s Vineyard a small group of fishermen began to exaggerate a tendency already existing in their speech. They did this seemingly subconsciously, in order to establish themselves as an independent social group with superior status to the despised summer visitors. A number of other islanders regarded this group as one which epitomised old virtues and desirable values, and subconsciously imitated the way its members talked. For these people, the new pronunciation was an innovation. As more and more people came to speak in the same way, the innovation gradually became the norm for those living on the island.

If I were placing bets, I’d guess that the rise in atheism is primarily a matter of picking up the low hanging fruit — people who were never strongly attached to their religion, had doubts about it, and now have an out and, as time passes, there are more natural atheists who are never born with even a weak tie to a particular faith tradition.  After most of the susceptibles get picked up (and I’ll bet there are still a lot of them out there), atheism’s rise would slow, and, stripped of their weakly committed members, churches would become more exaggerated and demonstrative.

I also wonder what kinds of trends the researches would observe if their model didn’t treat religion as a monolith.  If there are many small splinter sects (as there are today), each of them provides less social utility and connection than their parent conglomeration, and, in this model, each would be more vulnerable to extinction.  I wonder what the best way to build in churn (people moving from one religious sect to the other) would be, to recognize the fact that not everyone who leaves their religion becomes non-religious.