3 ms·

Unpopular opinion, but FiveThirtyEight constantly got the polls wrong and this isn't surprising. It gave people exaggerated confidence in their understanding of

Unpopular opinion, but FiveThirtyEight constantly got the polls wrong and this isn't surprising. It gave people exaggerated confidence in their understanding of the polling numbers. The amount of gaslighting I got from FiveThirtyEight readers about how Kamala wasn't guaranteed to win was astronomical.

kadoban 2y ago
> The amount of gaslighting I got from FiveThirtyEight readers about how Kamala wasn't guaranteed to win was astronomical. I can't find any way to read this other than that you think that Kamala was guaranteed to win, and are still angry that someone told you otherwise?
- mandolingual 2y ago
  From what I recall the main narrative was "it's neck and neck but we could be a single polling misstep from a blowout in either direction" which is about as safe a stance as you can take (understandable considering the blowback they got previously).
  - PaulHoule 2y ago
    More to the point the polls do not predict who wins the election because the election is won by the electoral college, not by the popular vote. Even if you got the popular vote exact you still couldn't predict the election accurately. What you have to do is simulate all the states and DC and run a monte carlo simulation and you will always get an equivocal answer if you do that.
    - JumpCrisscross 2y ago
      > polls do not predict who wins the election because the election is won by the electoral college, not by the popular vote Which is why any model worth its salt only uses national polls to predict state results. (State polls are more meaningful. But the headlines they generate aren't as nationally clickable.)
      - PaulHoule 2y ago
        Unfortunately there are too many states and not enough polls to make a model based on state-level polls so you have to infer state-level results based on something other than state-level polls.
        JumpCrisscross 2y ago
        > there are too many states and not enough polls to make a model based on state-level polls There aren’t that may states. You don’t need partisan polling in Vermont and Wyoming, for example. > you have to infer state-level results based on something other than state-level polls Yes. As I said. That is what you use national polls. “But the electoral college” isn’t a valid argument against election models. The reality is paucity of granular data and complexity of predicting swing states.
- TeaBrain 2y ago
  I think they probably meant the opposite. There's an statistics-illiterate subreddit for FiveThirtyEight, which largely refused to believe any poll that said that Kamala was going to lose, and heavily pushed any poll that reinforced the idea that Kamala would come out ahead.
  - techpineapple 2y ago
    [flagged]
    - TeaBrain 2y ago
      Thanks for catching that. Amusingly, it looks like we did make the same mistake. Completely unintentional on my part, but I've fixed it now. That first mentioned of "win" was meant to be "lose".
wobblyasp 2y ago
They aggregate polls. They weren't involved with polling
ljsprague 2y ago
My recollection is it was one of the few to allow for the possibility of Hillary losing.
neaden 2y ago
I think FiveThirtyEight's method is overrated but they basically put the last three elections as 50/50. Given how close the three of them were that seems fair enough.
- rurp 2y ago
  What? They had Biden at something like 90% to win. Hilary was around a 2-1 favorite which was vastly lower than almost anyone else.
- JumpCrisscross 2y ago
  > FiveThirtyEight's method is overrated but they basically put the last three elections as 50/50 You may be thinking of Nate Silver's Silver Bulletin [1]. He took the original 538 model with him when he left. The model 538 runs now is something they made up afterwards. [1] https://www.natesilver.net https://www.natesilver.net
- lotsofpulp 2y ago
  I don't consider losing all 7 swing states 50/50. If the Dems had a chance, presumably, they would have picked up at least 1 or 2? Also, all states increased their proportion of votes that Trump received.
  - rurp 2y ago
    Polling errors are highly correlated so it's not surprising if all states that are within a point or two break the same way. Nate's model had Trump sweeping the swing states as the single most likely outcome, with Harris sweeping all seven as the second most likely.
    - lotsofpulp 2y ago
      Sure, but when I hear 50/50, I expect the result will be nearly tied. Like a sports game. 312 electoral college votes versus 226 electoral college votes does not seem like it was a 50/50 game. Of course, this isn't a linear game where scores are alternating, so my line of thought isn't completely fair.
dragonwriter 2y ago
> Unpopular opinion, but FiveThirtyEight constantly got the polls wrong and this isn't surprising. 538 almost never got the polls wrong, they simply reported the results the pollsters got. But you probably meant something else was wrong, and not their reporting of the polls. Like, maybe, the poll-based predictions that the polls supported. But their poll-based predictions of individual results across all of the individual elections they predicted tended to be reasonably accurate in the Silver era, in that results predicted as, say, 70% chances were actually correct about 70% of the time, etc. In the post-Silver Morris period, I haven't seen any after-the-fact analysis, but the universe is a lot smaller, so you'd also expect the actual results on that limited universe to be less accurate in representing the long-term accuracy of the modelling approach. The main problem, though, is that people want predictions to have probability 1 or 0, and don't deal well with things in between.
eviks 2y ago
That's a popular opinion, but what does it have to do with this piece of news? They weren't closed it because they got the polls wrong?

kadoban 2y ago

> The amount of gaslighting I got from FiveThirtyEight readers about how Kamala wasn't guaranteed to win was astronomical. I can't find any way to read this other than that you think that Kamala was guaranteed to win, and are still angry that someone told you otherwise?

mandolingual 2y ago
From what I recall the main narrative was "it's neck and neck but we could be a single polling misstep from a blowout in either direction" which is about as safe a stance as you can take (understandable considering the blowback they got previously).
- PaulHoule 2y ago
  More to the point the polls do not predict who wins the election because the election is won by the electoral college, not by the popular vote. Even if you got the popular vote exact you still couldn't predict the election accurately. What you have to do is simulate all the states and DC and run a monte carlo simulation and you will always get an equivocal answer if you do that.
  - JumpCrisscross 2y ago
    > polls do not predict who wins the election because the election is won by the electoral college, not by the popular vote Which is why any model worth its salt only uses national polls to predict state results. (State polls are more meaningful. But the headlines they generate aren't as nationally clickable.)
    - PaulHoule 2y ago
      Unfortunately there are too many states and not enough polls to make a model based on state-level polls so you have to infer state-level results based on something other than state-level polls.
      - JumpCrisscross 2y ago
        > there are too many states and not enough polls to make a model based on state-level polls There aren’t that may states. You don’t need partisan polling in Vermont and Wyoming, for example. > you have to infer state-level results based on something other than state-level polls Yes. As I said. That is what you use national polls. “But the electoral college” isn’t a valid argument against election models. The reality is paucity of granular data and complexity of predicting swing states.
TeaBrain 2y ago
I think they probably meant the opposite. There's an statistics-illiterate subreddit for FiveThirtyEight, which largely refused to believe any poll that said that Kamala was going to lose, and heavily pushed any poll that reinforced the idea that Kamala would come out ahead.
- techpineapple 2y ago
  [flagged]
  - TeaBrain 2y ago
    Thanks for catching that. Amusingly, it looks like we did make the same mistake. Completely unintentional on my part, but I've fixed it now. That first mentioned of "win" was meant to be "lose".

wobblyasp 2y ago

They aggregate polls. They weren't involved with polling

ljsprague 2y ago

My recollection is it was one of the few to allow for the possibility of Hillary losing.

neaden 2y ago

I think FiveThirtyEight's method is overrated but they basically put the last three elections as 50/50. Given how close the three of them were that seems fair enough.

rurp 2y ago
What? They had Biden at something like 90% to win. Hilary was around a 2-1 favorite which was vastly lower than almost anyone else.
JumpCrisscross 2y ago
> FiveThirtyEight's method is overrated but they basically put the last three elections as 50/50 You may be thinking of Nate Silver's Silver Bulletin [1]. He took the original 538 model with him when he left. The model 538 runs now is something they made up afterwards. [1] https://www.natesilver.net https://www.natesilver.net
lotsofpulp 2y ago
I don't consider losing all 7 swing states 50/50. If the Dems had a chance, presumably, they would have picked up at least 1 or 2? Also, all states increased their proportion of votes that Trump received.
- rurp 2y ago
  Polling errors are highly correlated so it's not surprising if all states that are within a point or two break the same way. Nate's model had Trump sweeping the swing states as the single most likely outcome, with Harris sweeping all seven as the second most likely.
  - lotsofpulp 2y ago
    Sure, but when I hear 50/50, I expect the result will be nearly tied. Like a sports game. 312 electoral college votes versus 226 electoral college votes does not seem like it was a 50/50 game. Of course, this isn't a linear game where scores are alternating, so my line of thought isn't completely fair.

dragonwriter 2y ago

> Unpopular opinion, but FiveThirtyEight constantly got the polls wrong and this isn't surprising. 538 almost never got the polls wrong, they simply reported the results the pollsters got. But you probably meant something else was wrong, and not their reporting of the polls. Like, maybe, the poll-based predictions that the polls supported. But their poll-based predictions of individual results across all of the individual elections they predicted tended to be reasonably accurate in the Silver era, in that results predicted as, say, 70% chances were actually correct about 70% of the time, etc. In the post-Silver Morris period, I haven't seen any after-the-fact analysis, but the universe is a lot smaller, so you'd also expect the actual results on that limited universe to be less accurate in representing the long-term accuracy of the modelling approach. The main problem, though, is that people want predictions to have probability 1 or 0, and don't deal well with things in between.

eviks 2y ago

That's a popular opinion, but what does it have to do with this piece of news? They weren't closed it because they got the polls wrong?