3 ms·

Nobody was “proved to be correct”, because the winner of a poll does not by itself show anything about the probability of that particular candidate having won.

martingoodson 6y ago
The probability of winning is based on the popular support for a candidate. The poll directly measures that support. How can it possibly not 'show anything about the probability of that particular candidate having won'?
- xupybd 6y ago
  I think you are right it can. But the last election is proof that the polls can fail to predict the outcome of an election.
- WoahNoun 6y ago
  >The probability of winning is based on the popular support for a candidate This isn't actually true. The probability of winning is based on aggregation of regional probabilities. The same national vote probability distribution can lead to very, very, different election probability distributions due to regional variation. The electoral college essentially guarantees that the national probability distribution is worthless for actually predicting who will be President.
- matthewmacleod 6y ago
  Let's say we have two candidates A and B for some kind of election. We analyse our polling and other data, and estimate there is a 90% chance of candidate A winning and a 10% chance of candidate B winning. If candidate B then wins, it does not mean that our analysis was "proved to be [in]correct". By itself, it doesn't actually say anything about the quality of our analysis. After all, we explicitly pointed out that this was a possibility, and it would be strange to argue "your analysis said this might happen, and then it did, so your analysis was incorrect". There's just not enough information to draw any conclusions.
  - martingoodson 6y ago
    If one candidate won 1000 consecutive polls would it also tell you nothing about your estimate? This is obviously absurd: of course it would. How about 100 times? 10 times? 2 times? At what point does evidence cease to 'say anything about the quality of our analysis'? The answer is never. Every datapoint can be used to update your priors according to bayesian statistics.
    - WoahNoun 6y ago
      As I said in my other comment that you chose to ignore, the probability of winning national popular vote does not indicate the probability of who will be President.
      - martingoodson 6y ago
        It's not that I chose to ignore you. I just didn't find your comment interesting from the statistical perspective.
        WoahNoun 6y ago
        You don't find aggregation of regional deviation to national deviation interesting from a statistical perspective? Odd. Especially for someone who titles them-self "Chief Scientist"
        martingoodson 6y ago
        This is hacker news: I'm allowed to be more interested in bayesian statistics than your country's electoral system.
        WoahNoun 6y ago
        >This is hacker news: I'm allowed to be more interested in bayesian statistics than your country's electoral system. Aren't regional probabilities just conditions on the national distribution? Building the national Presidential election distribution from the conditioned distributions versus just using the sampled national distribution is bayesian statistics. >The probability of winning is based on the popular support for a candidate This was the argument you made. And I don't see how this is Bayesian vs frequency when all I am saying is that the same national popular vote distribution can have large variance in regional conditional distributions which leads to large variance in election outcome distribution due to the electoral college. Sounds pretty bayesian to me.
aaaaardvark 6y ago
I think Nate's _interpretation_ was proved correct, and in the vein of "all models are wrong, some were useful," his proved useful (or would have if people were taking it more seriously) in both forecasting a more uncertain result as well as ancillary things like showing which states were"tipping point states" e.g. where should you put your money if you want to win.
- aaaaardvark 6y ago
  Or rather, his interpretation was supported by the results, not proved
dap 6y ago
What does it mean to predict the probability of an event that can only possibly be tested once? Isn’t the prediction unfalsifiable?
- panarky 6y ago
  Sure, if the forecaster and the model were only used one time. But in 538's case they use the similar methods to forecast many individual contests like presidential primaries, presidential elections, senate elections, house elections, governor elections, club soccer, college football, March Madness men and women, MLB-NBA-NFL games and playoffs, etc. Multiple-year track records over many events so you can compare their forecasts to actual results over time. Get the data and compute your own error rate here: https://github.com/fivethirtyeight/checking-our-work-data https://github.com/fivethirtyeight/checking-our-work-data
  - dap 6y ago
    All of those things are very different though. It’s possible you could be excellent at predicting Senate races and terrible at the electoral college, right? And you only get one data point every 4 years.
- dllthomas 6y ago
  There's a philosophical side to your question that I am not going to engage with, but for the practical you might be interested in this: https://en.m.wikipedia.org/wiki/Scoring_rule https://en.m.wikipedia.org/wiki/Scoring_rule

martingoodson 6y ago

The probability of winning is based on the popular support for a candidate. The poll directly measures that support. How can it possibly not 'show anything about the probability of that particular candidate having won'?

xupybd 6y ago
I think you are right it can. But the last election is proof that the polls can fail to predict the outcome of an election.
WoahNoun 6y ago
>The probability of winning is based on the popular support for a candidate This isn't actually true. The probability of winning is based on aggregation of regional probabilities. The same national vote probability distribution can lead to very, very, different election probability distributions due to regional variation. The electoral college essentially guarantees that the national probability distribution is worthless for actually predicting who will be President.
matthewmacleod 6y ago
Let's say we have two candidates A and B for some kind of election. We analyse our polling and other data, and estimate there is a 90% chance of candidate A winning and a 10% chance of candidate B winning. If candidate B then wins, it does not mean that our analysis was "proved to be [in]correct". By itself, it doesn't actually say anything about the quality of our analysis. After all, we explicitly pointed out that this was a possibility, and it would be strange to argue "your analysis said this might happen, and then it did, so your analysis was incorrect". There's just not enough information to draw any conclusions.
- martingoodson 6y ago
  If one candidate won 1000 consecutive polls would it also tell you nothing about your estimate? This is obviously absurd: of course it would. How about 100 times? 10 times? 2 times? At what point does evidence cease to 'say anything about the quality of our analysis'? The answer is never. Every datapoint can be used to update your priors according to bayesian statistics.
  - WoahNoun 6y ago
    As I said in my other comment that you chose to ignore, the probability of winning national popular vote does not indicate the probability of who will be President.
    - martingoodson 6y ago
      It's not that I chose to ignore you. I just didn't find your comment interesting from the statistical perspective.
      - WoahNoun 6y ago
        You don't find aggregation of regional deviation to national deviation interesting from a statistical perspective? Odd. Especially for someone who titles them-self "Chief Scientist"
        martingoodson 6y ago
        This is hacker news: I'm allowed to be more interested in bayesian statistics than your country's electoral system.
        WoahNoun 6y ago
        >This is hacker news: I'm allowed to be more interested in bayesian statistics than your country's electoral system. Aren't regional probabilities just conditions on the national distribution? Building the national Presidential election distribution from the conditioned distributions versus just using the sampled national distribution is bayesian statistics. >The probability of winning is based on the popular support for a candidate This was the argument you made. And I don't see how this is Bayesian vs frequency when all I am saying is that the same national popular vote distribution can have large variance in regional conditional distributions which leads to large variance in election outcome distribution due to the electoral college. Sounds pretty bayesian to me.

aaaaardvark 6y ago

I think Nate's _interpretation_ was proved correct, and in the vein of "all models are wrong, some were useful," his proved useful (or would have if people were taking it more seriously) in both forecasting a more uncertain result as well as ancillary things like showing which states were"tipping point states" e.g. where should you put your money if you want to win.

aaaaardvark 6y ago
Or rather, his interpretation was supported by the results, not proved

dap 6y ago

What does it mean to predict the probability of an event that can only possibly be tested once? Isn’t the prediction unfalsifiable?

panarky 6y ago
Sure, if the forecaster and the model were only used one time. But in 538's case they use the similar methods to forecast many individual contests like presidential primaries, presidential elections, senate elections, house elections, governor elections, club soccer, college football, March Madness men and women, MLB-NBA-NFL games and playoffs, etc. Multiple-year track records over many events so you can compare their forecasts to actual results over time. Get the data and compute your own error rate here: https://github.com/fivethirtyeight/checking-our-work-data https://github.com/fivethirtyeight/checking-our-work-data
- dap 6y ago
  All of those things are very different though. It’s possible you could be excellent at predicting Senate races and terrible at the electoral college, right? And you only get one data point every 4 years.
dllthomas 6y ago
There's a philosophical side to your question that I am not going to engage with, but for the practical you might be interested in this: https://en.m.wikipedia.org/wiki/Scoring_rule https://en.m.wikipedia.org/wiki/Scoring_rule