Which cities claim to be the most ‘Bible-minded’ not the same as which really are

The Barna Research Group is part of my evangelical Christian world, which tends to give them both an advantage and a disadvantage when studying American religion.

Barna’s intimate familiarity and experience with the evangelical world gives them a deeper appreciation of some of the nuances of evangelical culture, but at the same time it makes them a bit less objective and sometimes a bit too much a part of the home team.

Barna’s research is mostly survey-driven, and they don’t always do a good job at distinguishing between what such surveys can and cannot tell us. Consider their recent report on the most and the least “biblicallly minded” cities in America.

What’s the scientific unit of measurement for biblical-mindedness? Well, there isn’t one, of course, so Barna cobbled a metric together based on “highest combined levels of regular Bible reading and belief in the Bible’s accuracy.”

OK, then, what’s the scientific unit of measurement for “levels of regular Bible reading”? Well, apparently, they just asked people how often they read their Bibles. Ditto for quantifying the unquantifiable and murkily outlined “belief in the Bible’s accuracy.”

Such surveys tell us something, but they cannot tell us what Barna’s report claims to tell us. If people in Knoxville, Shreveport and Chattanooga say they read the Bible regularly at a higher percentage than people in Providence, Albany and Burlington self-report such behavior, that suggests something about those cities, but it’s not a reliable measure of actual Bible-reading behavior. It tells us about the felt expectations and social obligations of residents of those communities, but that won’t work as an indicator of whether or not people are truly living up to such perceived expectations.

All that we can know for sure from such a survey is that people in Knoxville are more likely to tell a pollster that they read the Bible regularly. That might possibly correlate to higher levels of actual, regular Bible-reading, or it might just indicate that living in Knoxville carries a higher level of the sense that regular Bible-reading is a commendable activity that one ought to report to pollsters. I would guess that the latter is more likely.

Imagine a similar survey inquiring about “levels of regular exercise.” It would be a useful gauge of various cities’ attitudes toward exercise, revealing which communities experience a higher level of expectation or social pressure, or which places seem to instill a greater sense of the value and the obligation to exercise regularly. But it would be useless as a measurement of actual exercise habits and still more useless as a measure of physical fitness.

To measure or evaluate actual fitness or actual exercise, one would need to do something other than ask people about their habits, trusting that their responses were wholly accurate and truthful.

That’s not to say that people consciously lie about things like how often they exercise or how often they read the Bible — although that can happen, too, obviously. But I think questions about such behaviors are more like to produce aspirational answers than objectively descriptive answers. Respondents supply answers, in other words, that reflect the kind of behavior they feel like they ought to be demonstrating more than their actual behavior.

So how can we measure actual Bible-reading practices as opposed to only measuring the level of self-reporting? I have no idea.

But it would be fascinating, if we had a method of doing so, to contrast a measure of such actual behavior with the measure of such claimed behavior that Barna provides. That would allow us to make another Top 10 list — the “Top 10 Most Hypocritical Cities About Bible Reading.”

We don’t have data about actual behavior that would allow us to create such a list, but we know who the top candidates would be: Knoxville, Shreveport and Chattanooga. Whatever else Barna’s surveys suggest about Providence, Albany and Burlington, those cities can at least know that they’re probably not hotbeds of hypocrisy when it comes to “Bible-mindedness.” Without reliable data about actual behavior, Knoxville, et. al., due to their much-higher than normal levels of self-reported Bible-reading, have to be viewed with suspicion.

Good news, then, for Colorado Springs and Wheaton and Grand Rapids and all of the other evangelical hot-spots that failed to make Barna’s list of “Bible-minded” cities. This may not mean that they read their Bibles any less than Barna’s most-praised cities. It may just mean they’re more honest about such things.