This morning, the Gallup organization tried to explain why its polls of the 2012 presidential election turned out to be so wrong—Gallup’s final tracking poll indicated a 49 percent to 48 percent win for Mitt Romney, when in fact Barack Obama won by about 52-48. But the problem wasn’t just the final poll. Throughout the race, Gallup’s polls tended to be outliers that favored the Republican candidate. . . .
Gallup examined 20 issues and concluded that four were chiefly responsible for its having overestimated Romney’s response. They were:
1. Its “likely voter” model was flawed. It relied more heavily than other polls on someone’s past voting behavior and “thought given to the election,” a variable meant to predict who would and who wouldn’t vote that turned out to be not so accurate.
2. It blew the geography. By not interviewing enough people in certain regions of the country, specifically “the Eastern time zone within the Midwestern and Southern regions” and the “Pacific time zone within the Western region” (according to the report), it got a distorted picture of the race.
3. It messed up race and ethnicity. Gallup forced people to answer a series of yes/no questions about their race and ethnicity that didn’t yield an accurate picture, and then assumed minorities would make up a smaller share of the vote than they actually did.
4. It used a shortcut that didn’t work. For years, pollsters have relied on “random-digit dialing” (RDD) to reach both listed and unlisted numbers. This is expensive but, many pollsters believe, necessary since 45 percent of numbers are unlisted. Some recent academic research has indicated that it might be possible to get an accurate view of the electorate by scrapping RDD and combining (much cheaper) published phone numbers and cell phone users. (Huffington Post’s polling editor, Mark Blumenthal, has a thorough explainer here.) Gallup attempted this, and it didn’t work.
To me, another point also jumped out from this morning’s proceedings: Gallup said it hadn’t treated battleground states any differently than other states. Needless to say, the campaigns treated them much differently, running hundreds of millions of dollars of ads in those states that had a marked effect on voter sentiment.
Gallup’s Frank Newport explained that the point of the exercise was to improve performance in future races, and that its methodology was already being tweaked in light of the findings. “We’re in the process of modifying … how we conduct these models,” Newport said. “When the next presidential election rolls around … we think we’ll certainly be in a position to be at the accurate end of the spectrum, rather than in the uncomfortable position where we did not want to be this year, which is at the less accurate end of the spectrum.”
Ad-Free & Subscriber Log-In