9 ms·

They provided the probability distribution. The fact that you can’t handle math and need some sort of absolute certainty for a future event is not 538’s problem

by evgen 6y ago

They provided the probability distribution. The fact that you can’t handle math and need some sort of absolute certainty for a future event is not 538’s problem.

systemvoltage 6y ago
I am questioning - why even bother? It's not like we do elections every month to test out their probability distribution against empirical data. The distribution collapses into a binary outcome at the end. I have a dice. I claim the distribution is of equal outcome for each side. Well...we don't get to test the dice more than once. 1 sample size does not prove that the 538's predictions were right (or wrong). Thanks for assuming I can't do math, no way to argue with someone but I am actually pretty bad at it. :-)
- evgen 6y ago
  Everyone is bad at probability and statistical distributions, not just you. The problem with modeling elections is that there are so few of them and the data is very noisy and until quite recently rather suspect. Let's not pretend that this was a normal election, either in the candidates running or in the manner in which the campaign and election was conducted. As to the question of why bother, it is because bad polling is better than no polling at all. Campaigns are now multi billion dollar enterprises managing tens of thousands of temporary employees for the creation of a product that will only be sold once and in 18+ months from when they start the process. Any data is better than nothing. The fact that the public has become obsessed with polls is probably due to the ongoing nationalization of politics.
listenallyall 6y ago
it's not one single election -- it's consistent failure over multiple state elections, by large margins, all in the same direction -- which falls beyond any reasonable probability
- roenxi 6y ago
  I don't think much of evgen's unreasonable personal attack. But 538 isn't necessarily claiming that the per-state error will be normally distributed around their predictions. I don't know the specifics of their model, but probably they are claiming "with these polls, the probability of this outcome is...". The polls being consistently biased doesn't tell us much about 538s model. They said Biden would almost surely win and despite a massive surprise in favour of Trump, Biden won. And even if Biden lost, 10% upsets in presidential are expected to happen once every 10 elections like this one.
  - listenallyall 6y ago
    > But 538 isn't necessarily claiming that the per-state error will be normally distributed around their predictions. If per-state error isn't normally distributed, that's evidence of bias, or bad polling.
    - roenxi 6y ago
      Yep. But 538 does not run polls.
      - listenallyall 6y ago
        The inputs it uses to spit out probabilities is known to be bad. Any scientist or researcher who claimed to get valid results from known bad inputs would be ridiculed.
        roenxi 6y ago
        Science does not require an unbiased sample from a normal distribution to work. Bias is a technical term that the field of statistics is very comfortable working with. Scientists can also often get good results out of biased inputs. 538 has corrections for bias already. They seem to have worked in this instance - I repeat myself but: massive surprise, Biden still president.
defen 6y ago
Here is a guy who is fairly good at math saying that the polls did in fact fail: https://statmodeling.stat.columbia.edu/2020/11/04/dont-kid-yourself-the-polls-messed-up-and-that-would-be-the-case-even-wed-forecasted-biden-losing-florida-and-only-barely-winning-the-electoral-college/ https://statmodeling.stat.columbia.edu/2020/11/04/dont-kid-y... https://statmodeling.stat.columbia.edu/2020/11/07/what-would-would-mean-to-really-take-seriously-the-idea-that-our-forecast-probabilities-were-too-far-from-50/ https://statmodeling.stat.columbia.edu/2020/11/07/what-would...
- evgen 6y ago
  Ah yes. Mr 'let me tell you why Nate is wrong' Gelman, who is now Mr 'let me tell you why the fact that I missed bigger than Nate is not my fault and in fact is entirely the fault of these other people' Gelman. Forgive me if I find his excuses laughable, but I guess if it makes him feel better about himself we can humour him. He even manages to choke his first rant by missing once again on EV and vote percentages.
  - defen 6y ago
    Whatever your complaint about Gelman is, I don't think "he can't handle math" is very convincing.
- Karrot_Kream 6y ago
  That's a bit strong for what Gelman said. I'm a big fan of Gelman (and learned from his books!), but he specifically mentioned that both Gelman et al's Model and 538's Model did indeed capture the outcomes in their probability distributions, but that to improve performance going forward it was much better to predict closer to the median than closer to the tails. (And funny enough, Gelman gave 538 some grief earlier on making a model with very wide tails.) This is a nuanced but very fair criticism, and taking a Twitter-style summary of it I think is overly reductionist.

by evgen 6y ago

They provided the probability distribution. The fact that you can’t handle math and need some sort of absolute certainty for a future event is not 538’s problem.

systemvoltage 6y ago

I am questioning - why even bother? It's not like we do elections every month to test out their probability distribution against empirical data. The distribution collapses into a binary outcome at the end. I have a dice. I claim the distribution is of equal outcome for each side. Well...we don't get to test the dice more than once. 1 sample size does not prove that the 538's predictions were right (or wrong). Thanks for assuming I can't do math, no way to argue with someone but I am actually pretty bad at it. :-)

evgen 6y ago
Everyone is bad at probability and statistical distributions, not just you. The problem with modeling elections is that there are so few of them and the data is very noisy and until quite recently rather suspect. Let's not pretend that this was a normal election, either in the candidates running or in the manner in which the campaign and election was conducted. As to the question of why bother, it is because bad polling is better than no polling at all. Campaigns are now multi billion dollar enterprises managing tens of thousands of temporary employees for the creation of a product that will only be sold once and in 18+ months from when they start the process. Any data is better than nothing. The fact that the public has become obsessed with polls is probably due to the ongoing nationalization of politics.

listenallyall 6y ago

it's not one single election -- it's consistent failure over multiple state elections, by large margins, all in the same direction -- which falls beyond any reasonable probability

roenxi 6y ago
I don't think much of evgen's unreasonable personal attack. But 538 isn't necessarily claiming that the per-state error will be normally distributed around their predictions. I don't know the specifics of their model, but probably they are claiming "with these polls, the probability of this outcome is...". The polls being consistently biased doesn't tell us much about 538s model. They said Biden would almost surely win and despite a massive surprise in favour of Trump, Biden won. And even if Biden lost, 10% upsets in presidential are expected to happen once every 10 elections like this one.
- listenallyall 6y ago
  > But 538 isn't necessarily claiming that the per-state error will be normally distributed around their predictions. If per-state error isn't normally distributed, that's evidence of bias, or bad polling.
  - roenxi 6y ago
    Yep. But 538 does not run polls.
    - listenallyall 6y ago
      The inputs it uses to spit out probabilities is known to be bad. Any scientist or researcher who claimed to get valid results from known bad inputs would be ridiculed.
      - roenxi 6y ago
        Science does not require an unbiased sample from a normal distribution to work. Bias is a technical term that the field of statistics is very comfortable working with. Scientists can also often get good results out of biased inputs. 538 has corrections for bias already. They seem to have worked in this instance - I repeat myself but: massive surprise, Biden still president.

defen 6y ago

Here is a guy who is fairly good at math saying that the polls did in fact fail: https://statmodeling.stat.columbia.edu/2020/11/04/dont-kid-yourself-the-polls-messed-up-and-that-would-be-the-case-even-wed-forecasted-biden-losing-florida-and-only-barely-winning-the-electoral-college/ https://statmodeling.stat.columbia.edu/2020/11/04/dont-kid-y... https://statmodeling.stat.columbia.edu/2020/11/07/what-would-would-mean-to-really-take-seriously-the-idea-that-our-forecast-probabilities-were-too-far-from-50/ https://statmodeling.stat.columbia.edu/2020/11/07/what-would...

evgen 6y ago
Ah yes. Mr 'let me tell you why Nate is wrong' Gelman, who is now Mr 'let me tell you why the fact that I missed bigger than Nate is not my fault and in fact is entirely the fault of these other people' Gelman. Forgive me if I find his excuses laughable, but I guess if it makes him feel better about himself we can humour him. He even manages to choke his first rant by missing once again on EV and vote percentages.
- defen 6y ago
  Whatever your complaint about Gelman is, I don't think "he can't handle math" is very convincing.
Karrot_Kream 6y ago
That's a bit strong for what Gelman said. I'm a big fan of Gelman (and learned from his books!), but he specifically mentioned that both Gelman et al's Model and 538's Model did indeed capture the outcomes in their probability distributions, but that to improve performance going forward it was much better to predict closer to the median than closer to the tails. (And funny enough, Gelman gave 538 some grief earlier on making a model with very wide tails.) This is a nuanced but very fair criticism, and taking a Twitter-style summary of it I think is overly reductionist.