Today’s politics rely a great deal on polls. The problem is, election polling–which has had some spectacular failures lately–is faced with two huge problems:
(1) Cell phone usage has shot up, to the point that 47% of the population no longer has a land line. And robo-calling of cell phone numbers is illegal.
(2) The response rate to polls has plummeted. In the 1970s, the percentage of people who responded to pollster’s surveys was 80%. Today, it is 8%.
Thus, a recent national political poll conducted by NBC was based on just 236 responses!
From Cliff Zukin, What’s the Matter With Polling? – The New York Times:
Election polling is in near crisis, and we pollsters know. Two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically based, less well-tested techniques. To top it off, a perennial election polling problem, how to identify “likely voters,” has become even thornier.
In terms of speed, the growth of cellphones is like few innovations in our history. About 10 years ago, opinion researchers began taking seriously the threat that the advent of cellphones posed to our established practice of polling people by calling landline phone numbers generated at random. At that time, the National Health Interview Survey, a high-quality government survey conducted through in-home interviews, estimated that about 6 percent of the public used only cellphones. The N.H.I.S. estimate for the first half of 2014 found that this had grown to 43 percent, with another 17 percent “mostly” using cellphones. In other words, a landline-only sample conducted for the 2014 elections would miss about three-fifths of the American public, almost three times as many as it would have missed in 2008.
Since cellphones generally have separate exchanges from landlines, statisticians have solved the problem of finding them for our samples by using what we call “dual sampling frames” — separate random samples of cell and landline exchanges. The problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it’s not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers. Dialing manually for cellphones takes a great deal of paid interviewer time, and pollsters also compensate cellphone respondents with as much as $10 for their lost minutes.
THE best survey organizations, like the Pew Research Center, complete about two of the more expensive cellphone interviews for every one on a landline. For many organizations, this is a budget buster that leads to compromises in sampling and interviewing.
The second unsettling trend is the rapidly declining response rate. When I first started doing telephone surveys in New Jersey in the late 1970s, we considered an 80 percent response rate acceptable, and even then we worried if the 20 percent we missed were different in attitudes and behaviors than the 80 percent we got. Enter answering machines and other technologies. By 1997, Pew’s response rate was 36 percent, and the decline has accelerated. By 2014 the response rate had fallen to 8 percent. As Nate Silver of fivethirtyeight.com recently observed, “The problem is simple but daunting. The foundation of opinion research has historically been the ability to draw a random sample of the population. That’s become much harder to do.”
[Keep reading. . .]
For another good article on the subject, see this.
Ad-Free & Subscriber Log-In