On the Wrongness of Polls

THE POLLING INDUSTRY is a wreck, and should be blown up.”

What happened?  One explanation being offered is the “shy” Trump voter, the notion that people are embarrassed to admit to a pollster that they support the widely demonized president, lest they feel like a social pariah.

I think the problem goes deeper than that.  After all, the polls have also been wrong in missing conservative victories in other races, such as Brexit in the UK and the parliamentary elections in Australia.

I think survey results should include statistical data on the number of people who “declined to respond.”

I almost never respond to opinion polls of any kind.  Do you?  I don’t want to spend my precious time–“it won’t take more than 15 minutes”–being interrogated about my personal views by a stranger.

I wonder how many potential subjects just refuse to participate in the polls and if there is a correlation with conservative positions.

Actually, I used to participate in polls, thinking that my views might thereby be influential in the public square.  But the options given to me to choose from were seldom adequate to capture my positions.  When given four choices, or “agree” or “disagree,” it’s almost impossible to express complicated or nuanced opinions.  As for “how do you feel on a scale of one to ten,” my answers were almost always arbitrary, not really comparable to other subjects performing the same exercise, and thus not reliable at all.

And yet, though I almost never agree to take part in polls anymore, I did do an exit poll on Tuesday!  I had never been asked before, so I was kind of intrigued, wanting to see what it is like and to be involved in helping the media call a race.  I wasn’t much help.  I have a strong commitment to the secret ballot, so I never tell anyone how I vote.  But there were lots of other questions.  Again, the options didn’t fit with my thinking.  I was asked, “which issue is the most important to you?,” COVID-19, the economy, or health or safety?  Opposition to abortion was not a choice.  Nor was “moral breakdown” or “cultural collapse” or “opposition to postmodernism.”

The unreliability of polls has implications far beyond politics.  It calls into question the entire field of the social sciences, which rely to a  large degree on survey instruments and polling.  Already these disciplines are facing a crisis because it turns out so few of these studies are reproducible.  Different researchers get different results when they do the same experiment, which would indicate that human studies are not in accord with the scientific method.  Indeed, human beings–unlike inert chemicals and non-rational organisms–are not going to be predictably consistent or capable of being quantified.  This is because human beings, unlike other things scientists study, have agency.  It is becoming increasingly evident that the social sciences are not sciences at all.

I myself consult polling results and often blog about what a “study finds.”  When trying to understand cultural trends or how people view things, there is no other option than to try to find out what people say.  I think there can still be a use for these things, but now I wonder.  Perhaps researchers can refine their methods or somehow make their findings more reliable.  But I suspect there will always be major limitations to this kind of data, since human beings can change their minds and violate expectations.

Image by mohamed Hassan from Pixabay