Which Poll? (That is the question.)

I had a conversation today with Lukas, my son, about polls. I have only one principle: use the most reliable or accurate poll in the previous elections. As a result, from what I’ve seen, the Rasmussen and Pew polls have been the most accurate. So I tend to look to see where they are on the swing states. Which informs us that the election is up for grabs.

My son has another principle, or poll: Nate Silver.

Now I’m aware that Nate Silver, a poker player and baseball stat guy, who blogs now at the (ever balanced ;)) NYTimes. His blog is called FiveThirtyEight  is for some controversial. He uses a sophisticated algorithm and his numbers have been notably accurate in predicting elections. He has a lengthy post discussing the Pew and Rasmussen polls.

Steven Bronar puts Nate Silver to some tests, and finds him reliable but using a rather narrow range of data.

Nate Silver has said President Obama’s chances of winning are about 85%.

What say you? Which poll is your test? Are you betting against Nate Silver?

About Scot McKnight

Scot McKnight is a recognized authority on the New Testament, early Christianity, and the historical Jesus. McKnight, author of more than forty books, is the Professor of New Testament at Northern Seminary in Lombard, IL.

  • http://patheos.com John Longard

    The Rassmussen poll is a right-leaning poll.

  • http://bookwi.se Adam Shields

    The only poll I am worried about is on Tuesday. Otherwise I just don’t care. Polls are useful for politicians. For everyone else they are about the horse race which isn’t all that helpful for actually telling us about who would make a better president.

  • SteveSherwood

    I’m with your son. Nate Silver and his analysis of lots of polls and data seems to me to be a better source than just one poll (even if it is one that says what I want to hear).

  • jason

    I read Silver pretty regularly, and of all of the poll tracking sites think his work is the most impressive. Of course, I think I got a C in my last stats class some 20 years ago. So my opinion probably reflects my voting preference rather than any sophisticated assessment of his statistical model.

  • Jeremy

    Rasmussen seems to consistently skew to the Republican candidate. IIRC they regularly had McCain leading in 2008.

    Been reading Silver for a couple years now and find him quite insightful.

  • scotmcknight

    Jeremy, Yes, but the facts are that they were closest in the last election.

    That is, it might be that many of the other polls lean left.

  • Scott Wilson

    Rasmussen is proven to be the most accurate. Silver is neglecting certain data. Just like when we are in denial about things going on in our life we don’t want to face.

  • Tom F.

    You can read the comments in Bronar’s post where Bronar notes that Silver actually did end up (sort of) accounting for what Bronar was critiquing him for.

    Silver gives Romney a 15% chance to win. This 15% is basically only representative of one possibility; that the state polls are all biased against Mitt Romney. And mind you, not always biased against Republicans. Silver’s model accounts for this sort of bias. No, the 15% represents the possibility that all the polls are systematically biased against Romney above and beyond what they normally are for this election only.

    The relevant quote from Bronar is this: “In 2000, Al Gore outperformed state poll projections by an average of 2.14 percentage points. In 2004 George W. Bush outperformed state poll projections by an average of 1.52 percentage points, a swing of 3.66 percentage points in just 4 years. In addition, John McCain outperformed state polls by 0.70 percentage points in 2008. ”

    What Bronar is talking about is not systematic ideological bias, but rather an election specific factor that changes from election to election. Sometimes it benefits Democrats, sometimes Republicans. Right now, Romney is dependent on this factor going in his direction. And, based on Silver’s analysis, Romney’s chances of this going in his direction strongly enough are about 2:11.

  • Kyle J

    Silver’s algorithm is complex, but the principles it rests on are fairly simple:

    1) Use all the poll data, because more information is better than less information, but

    2) Give more weight to the polls that have proved most accurate in the past.

    If Silver is “wrong” (he doesn’t rule out Romney winning), it will only be because the vast preponderance of the polls were wrong (as Tom explains in more detail above).

  • Patrick

    I read an article that makes good logic to me on this issue. All polls have bias, except the candidate’s polls. They need the hard facts to arrange their cash flow and schedules such that they make the most bang for their buck so to speak.

    Watch where both camps campaigned the last weekend and Monday. IF both are in states one or the other normally wins, that tells you who is prevailing.

    For example, if Romney needs to be campaigning in Arizona or Georgia, Obama will win in a landslide. If Obama has to campaign in Pennsylvania, Minnesota or Wisconsin, he’s out. That makes more logic to me than standard polls.

    IMO, it doesn’t matter who gets elected, if they have a thimble full of wisdom they know Christ is the real ruler and He can stir the king’s heart and will, but, the church has a huge role to play in that, IMO. IF we’re half squared away, things will work out for us.

  • SFG

    Nate Silver was accurate in 49 of 50 states and all the senatorial races in 2008. You don’t get more accurate than that. And how can you not like a guy who got his start in baseball stats?

  • scotmcknight

    SFG, that’s what I’m taking about.

  • Jon Altman

    Rasmussen uses “robo polls,” which are not allowed to call cell phones. It consistently overestimates Republican strength. Nate Silver has been rock solid. Also, the Pew Poll now projects an Obama win.

  • http://grahambates.blogspot.com Graham

    I stay away from poles since I found http://www.intrade.com. People place money on who they think will win. Winners get $1 per “share” and losers get nothing. So far if you think Obama wins you have to put up $.63. Romney is $.37. Intrade has never been wrong since people put money on the line. If you think Romney will win, you can make a lot of money.

  • D. Vargas

    Rasmussen and Pew were not the most accurate in 2008.

    Obama won by 7.2 points, 52.9 to 45.7 percent (FEC verified).
    Rasmussen and Pew underestimated Obama with a 6-point spread, 52 to 46 percent.
    CNN and Ipsos came closer with a 7-point spread, 53 to 46 percent.

    Fordham University did a final accuracy analysis in which Rasmussen was beat out by eight other pollsters. http://www.fordham.edu/images/academics/graduate_schools/gsas/elections_and_campaign_/2008%20poll%20accuracy%20panagopoulos.pdf

  • Marshall

    If Obama has an 85% chance of winning, then in 100 random elections, Romney would win 15 times. That is to say, in this one particular random election, Silver could be quite perfectly correct about the odds being in Obama’s favor yet Romney wins. No contradiction.

    You gotta bet the long odds if you want to be a real punter.

  • NC

    I don’t have an opinion on the matter myself (I’m also tired of polls), but just thought I’d point you to Tim Stanley’s rather vitriolic critique of Nate Silver: http://blogs.telegraph.co.uk/news/timstanley/100186850/romney-obama-nate-silver-new-york-times/

  • Kenton

    Method and model are key when looking at poll data. How are they canvassing their input and with what weight do they bias it. Karl Rove has been insisting that the models being used are way out of whack, and the weight given to Democrats is too high. (In some cases they are weighting Democrat turnout to be higher than in ’08.)

    As for model, most all of these polls are canvassing to land lines. Who picks up their land line anymore? If anyone, it’s a low-income demographic that’s going to weighted more heavily to Democrats.

    I wouldn’t put money on either man winning based on the polls. There’s some indication of what’s happening based on the party affiliation of early voters and requests for absentee ballots, but I wouldn’t put money on that either. It’s going to be close tomorrow night, and for those of us who are political junkies, it’s going to be fun.

  • Kenton

    That should read “as for *method*” in the 2nd paragraph.

  • scotmcknight

    Kenton, all I’m suggesting is that we use the most reliable polls in the past.

  • Kenton

    Scot-

    Sure. And I readily agree that it’s usually a good idea to go with what’s been proven to be reliable in the past. But if the parameters change, then it’s time to be skeptical.

    In the days before caller-id and answering machines, calling phones at random was a good method of gathering input. That’s still true the elderly, but folks in mid-life avoid taking calls from people they don’t know. And 1st and 2nd time voters don’t even know that they can talk on phones. :)

    Do we really thing polling by phone is still accurate?

  • Bill

    Has anyone looked at this site:

    http://unskewedpolls.com/

  • CGC

    Hi Everyone,
    Well, the polls seem to be all over the map (forgive the pun). I did hear a fascinating discussion by a proffesor that the better question is not who are you going to vote for but who do you think is going to win the election? He suggest that the perception of the majority on who will win a election has been correct for the past 60 years of elections. If that is true (?), then the latest Gallup pole shows the strong majority of people perceiving an Obama win. All I can say when it comes to the mainstream media and television versus talk radio, one of these groups is lying. I guess we will find out who is more telling the truth after the election is over.

  • SteveSherwood

    Gallup and Rasmussen now have moved to a predicted draw. I think it makes for more hits to your site and eyes watching your channel if you say “too close to call” no matter what. Paul Krugman today compares that to saying a Yankee 1 run lead in the top of the 9th with 2 outs and Mariono Rivera pitching is a close game, but probably not “too close to call.” Statistically, the Yankees are at that point a pretty strong favorite.

  • CGC

    Hi Everyone,
    As I listen to some of the responses here, I just read Tim Suttle article which basically says, “3 ways to know your side is going to lose the election.” Here they are:
    1. Don’t believe the public polls.
    2. We are going to change the nature of the electorate.
    3. The only poll that counts is election day.

    He stated, “When you hear these things, you know you are about to lose!”

  • Kenny Johnson

    I think any of the aggregation sites are going to provide more accurate numbers. Others like Real Clear Politics, Politico, Princeton Election Consortium, etc are doing similar aggregations with some different methodology.

    All of them show Obama ahead in the electoral college.

  • Kenny Johnson

    So, based on the aggregators, I’m betting on

    Obama 303 – 332 (Florida is very very close).

  • unapologetic catholic

    I hope you’ll reconsider Nate Silver’s work, now. His forecasts were accurate down to the the number of electoral votes, which are held by states in unequal amounts. To hit the electoral votes right you have to accurately forecast the right combination of states.

  • CGC

    Hi Everyone,
    It has been amazing how accurate some of you were. I will say, after hearing both sides of the election saying not only that their candidate was going to win big and the other side is lying and deceiving us, I can’t help but wonder now that the election is basically over if people will take a strong look at who they listened and believed were telling them many things that simply were not true in regards to how this election was going to turn out.


CLOSE | X

HIDE | X