Innumeracy on the Godbeat

In the summer of 1992, toy-company Mattel was criticized when their new Teen Talk Barbie included in her list of stock phrases, “Math class is tough!” The company offered to replace the doll for disgruntled customers, but they could have saved both time and money by simply rebranding the dolls as Journalist Barbie.

The fact that math class was tough is the reason many of us (major exception is math whiz M.Z. Hemingway) ended up in the media, working with words. But when it comes to their reporting, even journalists who can solve quadratic equations in their heads often have trouble with basic mathematical concepts.

Consider for example what I call the “Implied Percentage Headline.” These are headlines that imply the article will show that Factor A affects Factor B and C by X percent. A recent example is Matthew Brown’s article in the Deseret News titled,
Faith and work: Accommodating religion boosts morale and bottom line.” The implication is that Factor A (religion) affects Factor B (morale) and C (the bottom line) by X percent. But a closer look at the numbers reveals something doesn’t add up.

Brown attributes the bold claim of the headline to Joyce Dubensky, CEO of the Tanenbaum Center for Interreligious Understanding:

Dubensky likes to tell that story to underscore a point her organization stresses when training companies and organizations on embracing diversity: Accommodating the religious needs of a workforce can boost morale and the bottom line.

Dubensky said business is finally responding. She cited research by DiversityInc that found 78 percent of the organization’s Top 50 diversified companies now offer floating religious holidays to employees compared with 42 percent nine years ago, and 70 percent provide prayer rooms today compared with 32 percent eight years ago. Awareness by employers and employees alike may continue to increase as America’s religious landscape becomes more diverse and issues of religious freedom arise as a result.

Before putting numbers into a news article, especially ones involving religion-related arguments, every journalist should ask, “Is that a significant number?” If they don’t know the answer themselves, then it’s likely their readers won’t know either.

Take, for instance, the first number in this claim which has to do with sample size: 50 companies. Even if these companies were randomly selected, the sample size would be too small to make a statistically relevant assertion about corporations in America. But the fact that they are 50 companies (our of a survey of 893) handpicked for diversity criteria (including religious diversity) makes the sample all but meaningless.

Yet even if the sample size were significant, the comparison would be invalid since the same companies were not being surveyed each time. The surveys referred to – surveys in 2003, 2004, and 2013 – include different companies each years. The 50 companies are chosen from all companies that have 1,000+ employees and have taken the survey. Since the same companies are not being compared, this also makes the comparison all but meaningless.

But even if the sample size and the comparison weren’t meaningless, would that have made the numbers significant? Well, they certainly sound impressive. The first claims a change from 42 to 78 percent – a 36 percent increase – while the second claims a change from – a 38 percent increase. When we look at the numbers behind the percentages, though, the changes appears rather trivial: a shift in the number of companies offering offered floating religious holidays from 21 to 30 and a shift in the number who provide prayer rooms from 16 to 35.

In the U.S there are over 6 million employers, and 18,469 of those firms have more than 1,000 people. Does a change of 9 tell use anything about a trend in companies offering floating religious holidays?

(I don’t mean to pick on Mr. Brown. This sort of “assertion by numbers” can be found all the time. But because it occurs in stories about sports or business rather than on the Godbeat, we tend not to notice when it happens.)

The article also gives us second opportunity to ask, “Is that a significant number?”

[R]eligion has remained the proverbial elephant in the room at the office — a topic human resources people feared was too sensitive and personal to address from a policy standpoint and yet loaded with liability if someone was offended by an insensitive remark or unintended impact of a company practice.

“Faith has existed in the workplace for years, but what’s new is that only now are people doing something about it,” Dubensky said.

Employers have learned the hard way that ignoring the faith needs of their workers exposes them to labor enforcement penalties. Statistics from the U.S. Equal Employment Opportunity Commission show that from 1997 to 2012, religious discrimination complaints soared 123 percent, with a high of 4,151 complaints lodged in 2011.

A 123 percent increase? That has to be significant, right?

Again, let’s look at the numbers behind the percentages. The difference in the number of complaints from 1997 to 2012 was 2,442 complaints. However, in order to be able to make an apples-to-apples comparison we have to know the size of the labor force for both 1997 and 2012.

In 1997, 136,297,000 adults were in the labor force. That means that in 1997, approximately one out of every 80,000 workers (0.000012 percent of the workforce) filed a religious discrimination complaint. In 2012, there were 154,900,000 workers, an average of 2.1 complaints for every 80,000 workers.

The number of complaints certainly increased substantially, but as a percentage of the total workforce, the number of people who complained stayed relatively low. Perhaps that number is noteworthy to a statistician employed at the EEOC. For most employees and employers, though, the changes remains rather insignificant.

The problem with including such numbers in a news article is that they give the impression they are conveying important information about religious trends. Neither these figures nor any of the other numbers included, though, offer much support for the claim that religion boosts morale or the bottom line. At best what the data in the article shows is that if you don’t want to risk lowering worker morale or getting sued for discrimination, don’t offend the religious sensibilities of your employees.

What the article really should have been about can be found in the last lines:

Meantime, Dubensky predicts companies will continue take a proactive approach to addressing the religious needs of their employees.

“We are getting more and more calls and companies are looking for information on a range of things,” she said. “In the next 15 years you will see an explosion of accommodations.”

That’s the real story, buried within what is admittedly an interesting and worthy (if perhaps a bit innumerate) feature. I don’t blame Brown for framing the article the way he did, though. A journalist is likely to get more readers writing about “how to improve the bottom line” than writing about “religious accommodations.” And for the media, the number of readers of a story is the most significant number of them all.

Print Friendly

  • Thinkling

    Percentages are notoriously difficult to work with…a 100% increase followed by another 100% increase gives not a 200%, but a 300% increase. I never convert anything to percentages until the very last step of any quantitative reporting.

  • Jerry

    Innumeracy is indeed a problem – thanks for highlighting it. I wonder sometimes about what is taught to aspiring journalists. Basic numeric reasoning should be one topic. Logic and logical fallacies should be another topic.

  • n_coast
  • Titleman

    I certainly agree with your observations about sample size and the representativeness of a sample. These are things that reporters frequently misunderstand. But I fear you yourself have fallen into another common error in reporting: confusing percentage points with percentage change.

    Now, I’m not an expert at math and would welcome any correction to my comments here. The error appears in this statement: “The first claims a change from 42 to 78 percent – a 36 percent increase – while the second claims a change from – a 38 percent increase.”

    An example will clarify the issue:
    At Time 1, I have a group of 15 people, five of whom are women. Women constitute 33% of my group. Later, at Time 2, the group changes its composition and there are now 10 women instead of five, making the percentage of women in the group 66%. The *percentage point* difference between these two points in time is 33 (66-33), but the *percent change* is 100%.

    This is because you calculate percentage change by taking number of people (or whatever is being counted) at Time 1 and subtracting from it the number of people/things at Time 2, then dividing that result by the number at Time 1.

    So if there are 10 women at Time 2 and five at Time 1, I get (10-5)/5 = 1.0 * 100 = 100%.

    Notice that if you had twenty people in the groups, with the same number of women (Time 1 = 5, Time 2 = 10), the situation would be this:
    -25% women at Time 1, 50% at Time 2
    -Difference in percentage points from T1 to T2 is +25
    -Percentage change, however, is the same as in the first example: 100%!

    So a change from 16 to 35 would be a 118% increase, regardless of whether the total sample was 50 or 50,000. The percentage point difference, however, would change considerably.

    I hope you will see that this is not mere quantitative quibbling, but reflects a fundamental point about how change is reported.

    None of this detracts from your main point about the actual number of companies under consideration here being so small and unrepresentative that no generalizable conclusions can be drawn from the data.

  • n_coast

    Since the 50 companies were selected on the basis of their employee diversity, the correlation implies that religious accommodations attract a diversified work force. By itself, that does not justify any claims about morale and the bottom line. (The Deseret News article did not tell us what were the differences between 78 and 42 or 70 and 32.)

    Small numbers offer fertile ground for interpretation. For example the World Health Organization has released statistics relating to cancer risk for persons living in the area affected by contamination from the Fukushima nuclear power plants. The predicted risk of thyroid cancer for infant girls could be up to 70% higher over their lifetime. Anti-nuclear activists can shout about that number; however generally the risk of thyroid for females in Japan is about 0.75%; so the increased risk is about 1.26%.


CLOSE | X

HIDE | X