9 ms·

Statistics, probability, and Nate Silver

48 points by jwallaceparker 14y ago

hooande 14y ago
Honestly? It's cool to see a statistician getting media love. I imagine for most people he's like a real life character from Big Bang Theory. I've seen him on tv several times and he comes off as incredibly nerdy, but he's skilled at talking about statistics and making it relatable to the average person. In a way, he's a spokesman for statistical reasoning to the world. It's odd to see how much pushback he gets for saying "A candidate with a slight lead in October wins 75% of the time". This is a fact, easy to look up. But because the other side and the media don't like it, Nate is put into the position of defending common sense to America. And he does an admirable job. Hopefully he can find a way to use this for good outside of politics. Maybe he can educate people about statistics and probability in other ways. Starting a blog that correctly predicts the behavior of reality tv stars might be the best thing he could do for the cause of reason in this country.
- _neil 14y ago
  Like a Neil deGrasse Tyson for statistics.
  - stcredzero 14y ago
    Maybe there's a media empire that can be built from people like this? We've always had such people in the media, but our tools and technology hold more potential power to individuals than ever before.
    - thronemonkey 14y ago
      I've long thought it would be awesome if someone started a business along these lines: a news source driven by data analysis; like fivethirtyeight but for analysis of all sorts of news and current events, not just elections.
      - stcredzero 14y ago
        How about "Freakonomics News?" A reality show using the methods used by the campaigns to figure out the behavior of "cohorts" could be devastating. Marketers have known scary stuff about us based on personal data, even before the Internet. A show based on actuarial data and gene sequencing would be both informative and appeal to base instincts.
shardling 14y ago
>Predicting an election based on polls is an entirely different matter. The election will turn out one way or another. If the same people voted for President 100 times without an external factor interfering differently across samples, the outcome would be the same every time. I'm glad someone pointed this out, since I think it gets obscured a bit in most discussion of 538's statistics.
- deleted 14y ago
  [deleted]
- erichocean 14y ago
  I don't think people are that deterministic. There's still internal factors that could cause them to change their vote; not everyone has everything 100% decided when they step in the polling booth. But yeah, it'd be pretty close.
Aaronontheweb 14y ago
One thing that really bothers me: most of Nate Silver's supporters and detractors don't understand jack shit about his models. On the Democrat side of the table, many treated 538 like a source of religious prophecy and source of comfort - when truth be told Nate is just a smart statistician who developed a model that's accurately predicted the outcomes of three elections (the first being the Hillary / Obama primary.) On the Republican side of the table, major trends in national and state-level polling data were ignored because pundits believed that the pollsters were oversampling Democrats when the truth is that the Democratic base has expanded since 2008. Other legitimate statisticians, like the University of Colorado, predicted a Romney win and obviously their models were wrong. Scientists, statisticians, economists, and engineers shouldn't be worshiped as sources of incontrovertible truth - they're just humans trying to do the best objective, accurate job that they can!
- deleted 14y ago
  [deleted]
  - deleted 14y ago
    [deleted]
- SkyMarshal 14y ago
  >Scientists, statisticians, economists, and engineers shouldn't be worshiped as sources of incontrovertible truth - they're just humans trying to do the best objective, accurate job that they can! True, but they're a big step in the direction of progressively less wrong [1] from the traditional alternative punditry and 'gut' analysis [2]. If I were a movie producer I'd buy the rights to Moneyball 2, script it about this election, and recast the gossiping scouting staff as all the blowhards [3] and bs artists [4]. Nate Silver, Andrew Tannenbaum [5], Drew Linzer [6], Sam Wang [7] and the other modelers deserve a lot of credit for helping the public cut through the bullshit. [1]: http://serendip.brynmawr.edu/sci_cult/lesswrong/lesswrong/ http://serendip.brynmawr.edu/sci_cult/lesswrong/lesswrong/ [2]: http://blogs.wsj.com/peggynoonan/2012/11/05/monday-morning/ http://blogs.wsj.com/peggynoonan/2012/11/05/monday-morning/ [3]: http://www.theatlanticwire.com/politics/2012/10/people-who-cant-do-math-are-so-mad-nate-silver/58460/ http://www.theatlanticwire.com/politics/2012/10/people-who-c... [4]: http://drudgereport.com http://drudgereport.com [5]: http://electoral-vote.com http://electoral-vote.com [6]: http://votamatic.org http://votamatic.org [7]: http://election.princeton.edu/ http://election.princeton.edu/
oulipo 14y ago
I understand what the post is trying to convey, yet it is easy for someone to cast the election & football in an equivalent framework: increasing the number of polls one of the statistician uses in order to narrow the confidence interval is equivalent to increasing the number of measurements you make on the football players & their environment before the game in order to narrow the confidence interval on the match outcome.. in other words you can interpret the confidence in the case of the election just as in the case of the football game, by stating that: 1. if your mathematical model of the population is accurate (eg. for instance your population really is voting using a binomial law of a certain probability p_democrat of voting for the democrat candidate) 2. and if the measurements (polls or player states) you have made give you that at least k voters out of N polled would have voted democrat then you can consider all the (hypothetical) instances of a population which would have resulted in a same or higher number of democrat voters for this poll, and you can compute the confidence interval that gives you the probability that such a population would indeed elect a democrat president So you don't necessarily have to view it in terms of "not changing the population vote, but changing the precision of the data", you can also cast it in just the framework of the football prediction
maaku 14y ago
One additional point that doesn't get much play in the debate about Nate Silver's numbers: political science, like economics, is a soft-science. You cannot 100% accurately predict the election based on polls, just as you cannot accurately predict market movements based on economic indicators. This is because the system is made up of self-aware human actors that respond to meta-information like Nate Silver's numbers. Even if the polls were 100% accurate in measuring voter sentiments, as soon as they are published they are picked up by political operatives and become part of the spin machine which changes those sentiments. But if you want to use polling data, Nate Silver is demonstrating the least-bad way to do it.
- csense 14y ago
  > This is because the system is made up of self-aware human actors that respond to meta-information like Nate Silver's numbers. The fact that agents use polling information to make decisions doesn't necessarily mean the system won't reach equilibrium. But I do think it's conceivable that one candidate's predicted overwhelming chance of victory might have a way of becoming a self-fulfilling prophecy in a close election. Ron Paul supporters presumably prefer Romney to Obama. In an election where it was reported that the race was close, they'd vote for Romney. But if it was reported that an Obama win was basically guaranteed, they might be willing to vote for the Libertarian Party or write-in Ron Paul (who isn't even running) as a protest vote. I really think our democracy would be healthier if we switched to an instant runoff system [1], but the Democrats and Republicans have a very strong incentive to unite against anything that could strengthen third parties. [1] http://en.wikipedia.org/wiki/Instant-runoff_voting http://en.wikipedia.org/wiki/Instant-runoff_voting
  - maaku 14y ago
    > But I do think it's conceivable that one candidate's predicted overwhelming chance of victory might have a way of becoming a self-fulfilling prophecy in a close election. It also has the possibility for the opposite by lowering turnout for the "winning" candidate. Why vote? Nate Silver already called the election, etc. There are other ways in which polling data itself can influence an election. It appears that Nate Silver did an great job early on at predicting the effect of changing demographics in swing states like Florida--which ended up giving the election to Obama. Hindsight being 20/20, if the Romney team had paid more attention to that (and perhaps compromised on things like immigration reform), things could have gone very differently.
jules 14y ago
> There is internal chaos in the game that forces the probabilistic distribution. Predicting an election based on polls is an entirely different matter. The election will turn out one way or another. If the same people voted for President 100 times without an external factor interfering differently across samples, the outcome would be the same every time. This is a distinction without a difference. There is "internal chaos" in the game because we don't know its initial conditions exactly, just like in the elections. If we had precise knowledge about the particles in the ball, the field, and the players, we could theoretically predict the outcome with high certainty. Except for quantum mechanical effects, statistics isn't about true randomness, it's about modeling lack of knowledge.
- loup-vaillant 14y ago
  Nitpick: even for quantum mechanical effects. The equations have no randomness in them. As far as we know, the laws of physics are deterministic. It's just that we can't know in advance which Everett branch we will subjectively experience. We could (in principle) compute what proportion of Everett branches lead to what outcome, though. Probability is still in the mind. Not in the territory. http://lesswrong.com/lw/oj/probability_is_in_the_mind/ http://lesswrong.com/lw/oj/probability_is_in_the_mind/ (In case you didn't already spot me.)
  - jules 14y ago
    In a sense that's true, but Bell's inequalities show that there is something qualitatively different about quantum mechanical randomness. Bar some technicalities, they show that there isn't some hidden internal state that determines what outcome we experience, unlike with elections and baseball (assuming QM effects don't play a significant role in those). In this sense, QM is "truly random". That said, even though there is good experimental evidence for this, it is not 100% certain that the world actually works this way. http://en.wikipedia.org/wiki/Bells_theorem http://en.wikipedia.org/wiki/Bells_theorem
    - loup-vaillant 14y ago
      Nothing is 100% sure :-) http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ But yeah, as long as we didn't find complete laws of physics, we're bound to ignore something.

Honestly? It's cool to see a statistician getting media love. I imagine for most people he's like a real life character from Big Bang Theory. I've seen him on tv several times and he comes off as incredibly nerdy, but he's skilled at talking about statistics and making it relatable to the average person. In a way, he's a spokesman for statistical reasoning to the world. It's odd to see how much pushback he gets for saying "A candidate with a slight lead in October wins 75% of the time". This is a fact, easy to look up. But because the other side and the media don't like it, Nate is put into the position of defending common sense to America. And he does an admirable job. Hopefully he can find a way to use this for good outside of politics. Maybe he can educate people about statistics and probability in other ways. Starting a blog that correctly predicts the behavior of reality tv stars might be the best thing he could do for the cause of reason in this country.

_neil 14y ago
Like a Neil deGrasse Tyson for statistics.
- stcredzero 14y ago
  Maybe there's a media empire that can be built from people like this? We've always had such people in the media, but our tools and technology hold more potential power to individuals than ever before.
  - thronemonkey 14y ago
    I've long thought it would be awesome if someone started a business along these lines: a news source driven by data analysis; like fivethirtyeight but for analysis of all sorts of news and current events, not just elections.
    - stcredzero 14y ago
      How about "Freakonomics News?" A reality show using the methods used by the campaigns to figure out the behavior of "cohorts" could be devastating. Marketers have known scary stuff about us based on personal data, even before the Internet. A show based on actuarial data and gene sequencing would be both informative and appeal to base instincts.

shardling 14y ago

>Predicting an election based on polls is an entirely different matter. The election will turn out one way or another. If the same people voted for President 100 times without an external factor interfering differently across samples, the outcome would be the same every time. I'm glad someone pointed this out, since I think it gets obscured a bit in most discussion of 538's statistics.

deleted 14y ago
[deleted]
erichocean 14y ago
I don't think people are that deterministic. There's still internal factors that could cause them to change their vote; not everyone has everything 100% decided when they step in the polling booth. But yeah, it'd be pretty close.

Aaronontheweb 14y ago

One thing that really bothers me: most of Nate Silver's supporters and detractors don't understand jack shit about his models. On the Democrat side of the table, many treated 538 like a source of religious prophecy and source of comfort - when truth be told Nate is just a smart statistician who developed a model that's accurately predicted the outcomes of three elections (the first being the Hillary / Obama primary.) On the Republican side of the table, major trends in national and state-level polling data were ignored because pundits believed that the pollsters were oversampling Democrats when the truth is that the Democratic base has expanded since 2008. Other legitimate statisticians, like the University of Colorado, predicted a Romney win and obviously their models were wrong. Scientists, statisticians, economists, and engineers shouldn't be worshiped as sources of incontrovertible truth - they're just humans trying to do the best objective, accurate job that they can!

deleted 14y ago
[deleted]
- deleted 14y ago
  [deleted]
SkyMarshal 14y ago
>Scientists, statisticians, economists, and engineers shouldn't be worshiped as sources of incontrovertible truth - they're just humans trying to do the best objective, accurate job that they can! True, but they're a big step in the direction of progressively less wrong [1] from the traditional alternative punditry and 'gut' analysis [2]. If I were a movie producer I'd buy the rights to Moneyball 2, script it about this election, and recast the gossiping scouting staff as all the blowhards [3] and bs artists [4]. Nate Silver, Andrew Tannenbaum [5], Drew Linzer [6], Sam Wang [7] and the other modelers deserve a lot of credit for helping the public cut through the bullshit. [1]: http://serendip.brynmawr.edu/sci_cult/lesswrong/lesswrong/ http://serendip.brynmawr.edu/sci_cult/lesswrong/lesswrong/ [2]: http://blogs.wsj.com/peggynoonan/2012/11/05/monday-morning/ http://blogs.wsj.com/peggynoonan/2012/11/05/monday-morning/ [3]: http://www.theatlanticwire.com/politics/2012/10/people-who-cant-do-math-are-so-mad-nate-silver/58460/ http://www.theatlanticwire.com/politics/2012/10/people-who-c... [4]: http://drudgereport.com http://drudgereport.com [5]: http://electoral-vote.com http://electoral-vote.com [6]: http://votamatic.org http://votamatic.org [7]: http://election.princeton.edu/ http://election.princeton.edu/

oulipo 14y ago

I understand what the post is trying to convey, yet it is easy for someone to cast the election & football in an equivalent framework: increasing the number of polls one of the statistician uses in order to narrow the confidence interval is equivalent to increasing the number of measurements you make on the football players & their environment before the game in order to narrow the confidence interval on the match outcome.. in other words you can interpret the confidence in the case of the election just as in the case of the football game, by stating that: 1. if your mathematical model of the population is accurate (eg. for instance your population really is voting using a binomial law of a certain probability p_democrat of voting for the democrat candidate) 2. and if the measurements (polls or player states) you have made give you that at least k voters out of N polled would have voted democrat then you can consider all the (hypothetical) instances of a population which would have resulted in a same or higher number of democrat voters for this poll, and you can compute the confidence interval that gives you the probability that such a population would indeed elect a democrat president So you don't necessarily have to view it in terms of "not changing the population vote, but changing the precision of the data", you can also cast it in just the framework of the football prediction

maaku 14y ago

One additional point that doesn't get much play in the debate about Nate Silver's numbers: political science, like economics, is a soft-science. You cannot 100% accurately predict the election based on polls, just as you cannot accurately predict market movements based on economic indicators. This is because the system is made up of self-aware human actors that respond to meta-information like Nate Silver's numbers. Even if the polls were 100% accurate in measuring voter sentiments, as soon as they are published they are picked up by political operatives and become part of the spin machine which changes those sentiments. But if you want to use polling data, Nate Silver is demonstrating the least-bad way to do it.

csense 14y ago
> This is because the system is made up of self-aware human actors that respond to meta-information like Nate Silver's numbers. The fact that agents use polling information to make decisions doesn't necessarily mean the system won't reach equilibrium. But I do think it's conceivable that one candidate's predicted overwhelming chance of victory might have a way of becoming a self-fulfilling prophecy in a close election. Ron Paul supporters presumably prefer Romney to Obama. In an election where it was reported that the race was close, they'd vote for Romney. But if it was reported that an Obama win was basically guaranteed, they might be willing to vote for the Libertarian Party or write-in Ron Paul (who isn't even running) as a protest vote. I really think our democracy would be healthier if we switched to an instant runoff system [1], but the Democrats and Republicans have a very strong incentive to unite against anything that could strengthen third parties. [1] http://en.wikipedia.org/wiki/Instant-runoff_voting http://en.wikipedia.org/wiki/Instant-runoff_voting
- maaku 14y ago
  > But I do think it's conceivable that one candidate's predicted overwhelming chance of victory might have a way of becoming a self-fulfilling prophecy in a close election. It also has the possibility for the opposite by lowering turnout for the "winning" candidate. Why vote? Nate Silver already called the election, etc. There are other ways in which polling data itself can influence an election. It appears that Nate Silver did an great job early on at predicting the effect of changing demographics in swing states like Florida--which ended up giving the election to Obama. Hindsight being 20/20, if the Romney team had paid more attention to that (and perhaps compromised on things like immigration reform), things could have gone very differently.

jules 14y ago

> There is internal chaos in the game that forces the probabilistic distribution. Predicting an election based on polls is an entirely different matter. The election will turn out one way or another. If the same people voted for President 100 times without an external factor interfering differently across samples, the outcome would be the same every time. This is a distinction without a difference. There is "internal chaos" in the game because we don't know its initial conditions exactly, just like in the elections. If we had precise knowledge about the particles in the ball, the field, and the players, we could theoretically predict the outcome with high certainty. Except for quantum mechanical effects, statistics isn't about true randomness, it's about modeling lack of knowledge.

loup-vaillant 14y ago
Nitpick: even for quantum mechanical effects. The equations have no randomness in them. As far as we know, the laws of physics are deterministic. It's just that we can't know in advance which Everett branch we will subjectively experience. We could (in principle) compute what proportion of Everett branches lead to what outcome, though. Probability is still in the mind. Not in the territory. http://lesswrong.com/lw/oj/probability_is_in_the_mind/ http://lesswrong.com/lw/oj/probability_is_in_the_mind/ (In case you didn't already spot me.)
- jules 14y ago
  In a sense that's true, but Bell's inequalities show that there is something qualitatively different about quantum mechanical randomness. Bar some technicalities, they show that there isn't some hidden internal state that determines what outcome we experience, unlike with elections and baseball (assuming QM effects don't play a significant role in those). In this sense, QM is "truly random". That said, even though there is good experimental evidence for this, it is not 100% certain that the world actually works this way. http://en.wikipedia.org/wiki/Bells_theorem http://en.wikipedia.org/wiki/Bells_theorem
  - loup-vaillant 14y ago
    Nothing is 100% sure :-) http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ But yeah, as long as we didn't find complete laws of physics, we're bound to ignore something.