Christian researchers accused of misrepresentation

The Family Research Council, based in Washington DC, is “a Christian organization promoting the traditional family unit and the Judeo- Christian value system upon which it is built” (yep, they really do think that the idea of ‘family’ is a Jewish invention…). They have the usual values that we’ve come to expect from extremist Christianity.

Anyway, earlier this month they put out a study claiming that “children have fewer problems at school and home when they live with both biological parents and frequently attend religious services”. But it turns out that not only have they spun the data till it screams, but they also took a decidedly dodgy approach to presenting the facts.

For example, they claim that the study supports marriage, but in fact the survey data that they used in the study doesn’t say anything about the marital status of the parents! All it talks about is whether the care-givers are the biological parents, and not whether or not they are married. So that is an impossible conclusion to draw.

Furthermore, the effect of religious attendance was minuscule – far more important was family stability. This is something that the study authors themselves acknowledge (“One of the things that we say in the report is that the behavior problems were more closely related to the intact two-parent family than to a lack of religious participation, Zill said.”), although you don’t find any mention of it in the Family Research Council’s press release or associated publicity.

According to two independent academics from West Virginia, there are a number of other flaws both in the study and in the way that the Family Research Council has presented it. WVS Public Broadcasting reports:

Besides these inconsistencies, two researchers in West Virginia say the study does not stand up to academic scrutiny.

Joseph Scotti is a professor of clinical child psychology at West Virginia University. He says the study skews the data and doesn’t reach any meaningful conclusions.

Prof Scotti’s gripe is that the data have been presented in a way that makes tiny differences look much bigger than they actually are:

Scotti says the study presents data in ways that wouldn’t be considered valid by most social scientists. Charts are organized so even miniscule differences appear significant. There is also the question of whether any parent-child activity would help reduce behavioral problems.

Take a look at this chart, for example. The axes have been trimmed to exaggerate the differences between the bars. What’s more, in a proper scientific study, you would be given a statistical assessment (some kind of analysis of whether the differences are down to chance). With this study, you get no such assessment – probably because the differences are so small that they are just random noise.

There are more problems. Although attending religious service may be associated with somewhat better behaviour, it’s not at all clear that religion has anything to do with it. Just as likely, participating in any regular, structured activity (like sports) will have a similar effect.

“My first question would be is it the structured activity and being involved with activities that keep kids out of trouble, regardless of whether it’s religious or scouting or volunteering or whatever it is,” Scotti added.

The other academic quoted, Marybeth Beller (a political science professor at Marshall University) agrees with Scotti:

“Two things must be noted. The first is that the magnitude of that relationship is very small. It varies by two percent in some of their correllation and up to three percent in others. But a three percent difference is just that; it’s a very small magnitude,” she said.

But, she also goes on to point out another important fact: the authors don’t even attempt to take into account other factors that are known to affect the behaviour and academic achievement of children.

For example, children in poor families are underacheivers and also are more likely to live with non-biological parents. But perhaps it’s the poverty and deprivation that’s causing the underachievement, and the loss of one-or-other parent is neither here nor there.

This is not a new idea. I rapidly found a study published by the US Census Burea in 1998 which showed that, after accounting for socioeconomic factors, the effects of marital status dissappeared. To be fair, there’s a lot of research on this topic, and the exact effects of marriage are not clear (and probably vary according to culture anyway). But there’s not even an attempt to account for this sort of thing in the Family Research Council’s study.

Finally, even supposing that such an effect exists, what would be the take home message? Maybe it would be that we should encourage marital stability. But it might also be that we should be providing more support and social acceptance to the children of families whose parental bond has broken down.