5 ms·

Looks like we've not learned anything from 2016.

by kkotak 6y ago

rhacker 6y ago
That's what I came in to say. If it happens again I think we can conclude that at this point, the people voting and the people getting ahead looking polls (say on the internet, random calls, etc..) are not the same people.
- edmundsauto 6y ago
  Let's say the odds stay where they are, and Trump wins again. That would be hitting on a 30% chance in 2016 and hitting on a 20% chance in 2020. That's a 6% chance overall. 6% likely outcomes happen all the time. It would be a textbook example of "resulting" to draw a conclusion based on this.
  - T-hawk 6y ago
    [deleted]
    - rsynnott 6y ago
      > If the polls didn't match the votes in 2016, they probably won't again in 2020, for at least some of the same underlying reasons. That's true only if you assume perfectly spherical pollsters :) After a real or perceived polling miss (in reality, most polls weren't off by all that much in 2016), pollsters tend to change assumptions, sometimes overcompensating. Notably, after a moderate polling miss in the 2015 elections, which predicted a hung parliament where in fact the Tories managed a small majority, UK pollsters went on to overcompensate in the 2017 election, showing a Tory blow-out win whereas in fact they ended up with a hung parliament. (That one, incidentally, shows one danger of polls! The first poll allowed Cameron to safely promise a vote on the EU, because he assumed they'd be either out of government or in coalition with the libdems, who wouldn't allow it. In fact, they won and were forced to go through with it. The second one allowed Theresa May to call a snap election to gain seats so as to have enough spare MPs to be able to ignore the ERG and negotiate a semi-sensible Brexit. Instead, they lost seats, had to deal with the ERG and Unionists, and ultimately ended up with the current catastrophe. The current Brexit mess is at least in part due to polling misses.) In the US, pollsters have started to pay a lot more attention to education, which was a major predictor in the 2016 election.
  - deleted 6y ago
    [deleted]
  - rhacker 6y ago
    I am not saying that. I'm saying that the polling data is so flawed because it's not getting into a specific democratic that it's off by much wider margin. So much so that if Trump wins again it's because it was a 40% chance he won both 2016 and 2020, we just don't have the polling data to prove that number. So while you say 6% "happens all the time" - no it actually happens 6% of the time. But 40% for both terms would indicate more like he had a 70% last time and maybe another 60% this time. The pollsters can be correct for fucking California, but what good is that if they're super wrong because racist people in Alabama don't click or pick up the phone (or have no phone to begin with) or talk to pollsters.
    - edmundsauto 6y ago
      You would need more evidence to determine that. 2016 + 2020 isn't enough -- you'd need to actually show that this is happening, rather than using 2 results to back into your belief. To clarify, my intention was to state, "outcomes with 6% of odds happening will occur very frequently" - not relatively. A good hitter in baseball hits homers about 6% of the time. If they hit homers in 2 at bats, we would not have enough information to say much about their true talent.
- rsynnott 6y ago
  On a national level, the polls really did, well, not amazing, but fine, last time. 538's final polling average was Clinton+3.8, actual result was Clinton+2.1. That's not at all bad; you wouldn't call that an upset. The real problem was a lack of high-quality state-level polls, and in one or two cases major misses on what state polls there were; there wasn't all that much visibility on many states. And ultimately, the final result was so close that it was below the margin of error of any reasonable poll to detect. Even assuming the highest quality polls conducted in each state, every day, the best they'd have been able to say would have been, pretty much "It's 50/50", in retrospect.
  - nl 6y ago
    And the state level polling errors were highly correlated, which 538 called out in their pre-election coverage, but people ignored.
  - dragonwriter 6y ago
    > The real problem was a lack of high-quality state-level polls That wasn't even really a problem; the error in state polling was pretty consistent with the error in national polling; the problem with predictors other than 538 is they treated state variations from polling averages as independent, 538 correctly assessed them (based on past evidence) as highly correlated, which is why Trump had a nearly 1 in 3 chance in 538s forecast. The problem is people taking (honestly or just for the purpose of after-the-fact criticism) "1 in 3" to mean "absolutely won't happen".
verylittlemeat 6y ago
You're being downvoted but you're right. I'm starting to believe that sites like 538 feed off of the general public not understanding that a 20% chance to win is 1 in 5. Endless ink has been spilled about how the public does not understand statistics but very little effort has been made to communicate effectively. The whole thing seems like a navelgazing sideshow where the pollsters want to have it both ways. They want to claim their predictions are infallible but then when the public says they failed they want to backpedal with holier than thou "well, actually..." excuses.
- aw1621107 6y ago
  > Endless ink has been spilled about how the public does not understand statistics but very little effort has been made to communicate effectively. How, then, should statistics be appropriately communicated? > They want to claim their predictions are infallible but then when the public says they failed they want to backpedal with holier than thou "well, actually..." excuses. Do pollsters generally try to claim their numbers are infallible? If anything, the fact that margins of error are included in the results would seem to imply the opposite.
  - verylittlemeat 6y ago
    I don't buy that they're trying to communicate in good faith. I remember very clearly sources like this* in 2016. There was so much hubris and minimizing of Trump that him having a 30% chance to win was like an irrelevant footnote to Hillary's coming coronation. *https://pbs.twimg.com/media/Cwc-j6YXUAEIMaM.jpg https://pbs.twimg.com/media/Cwc-j6YXUAEIMaM.jpg The pollsters take too much credit when they're right and deflect too much blame when they're wrong. They need to do a better job of being humble and stop trying to pass themselves off as apolitical number crunchers just giving us the facts.
    - aw1621107 6y ago
      > I don't buy that they're trying to communicate in good faith. What would "good faith" communication look like, then? > I remember very clearly sources like this* in 2016. Can you elaborate on what exactly is wrong with that source? (That isn't a pollster, too; that's a (meta?)analysis/aggregation, but that's a relatively minor nitpick) > They need to do a better job of being humble and stop trying to pass themselves off as apolitical number crunchers just giving us the facts. Are they themselves claiming that they are "giving us the facts", or is that how they are being represented by others? And are you talking about actual polls, or about predictions based on said polls?

Looks like we've not learned anything from 2016.

by kkotak 6y ago

Looks like we've not learned anything from 2016.

That's what I came in to say. If it happens again I think we can conclude that at this point, the people voting and the people getting ahead looking polls (say on the internet, random calls, etc..) are not the same people.

edmundsauto 6y ago
Let's say the odds stay where they are, and Trump wins again. That would be hitting on a 30% chance in 2016 and hitting on a 20% chance in 2020. That's a 6% chance overall. 6% likely outcomes happen all the time. It would be a textbook example of "resulting" to draw a conclusion based on this.
- T-hawk 6y ago
  [deleted]
  - rsynnott 6y ago
    > If the polls didn't match the votes in 2016, they probably won't again in 2020, for at least some of the same underlying reasons. That's true only if you assume perfectly spherical pollsters :) After a real or perceived polling miss (in reality, most polls weren't off by all that much in 2016), pollsters tend to change assumptions, sometimes overcompensating. Notably, after a moderate polling miss in the 2015 elections, which predicted a hung parliament where in fact the Tories managed a small majority, UK pollsters went on to overcompensate in the 2017 election, showing a Tory blow-out win whereas in fact they ended up with a hung parliament. (That one, incidentally, shows one danger of polls! The first poll allowed Cameron to safely promise a vote on the EU, because he assumed they'd be either out of government or in coalition with the libdems, who wouldn't allow it. In fact, they won and were forced to go through with it. The second one allowed Theresa May to call a snap election to gain seats so as to have enough spare MPs to be able to ignore the ERG and negotiate a semi-sensible Brexit. Instead, they lost seats, had to deal with the ERG and Unionists, and ultimately ended up with the current catastrophe. The current Brexit mess is at least in part due to polling misses.) In the US, pollsters have started to pay a lot more attention to education, which was a major predictor in the 2016 election.
- deleted 6y ago
  [deleted]
- rhacker 6y ago
  I am not saying that. I'm saying that the polling data is so flawed because it's not getting into a specific democratic that it's off by much wider margin. So much so that if Trump wins again it's because it was a 40% chance he won both 2016 and 2020, we just don't have the polling data to prove that number. So while you say 6% "happens all the time" - no it actually happens 6% of the time. But 40% for both terms would indicate more like he had a 70% last time and maybe another 60% this time. The pollsters can be correct for fucking California, but what good is that if they're super wrong because racist people in Alabama don't click or pick up the phone (or have no phone to begin with) or talk to pollsters.
  - edmundsauto 6y ago
    You would need more evidence to determine that. 2016 + 2020 isn't enough -- you'd need to actually show that this is happening, rather than using 2 results to back into your belief. To clarify, my intention was to state, "outcomes with 6% of odds happening will occur very frequently" - not relatively. A good hitter in baseball hits homers about 6% of the time. If they hit homers in 2 at bats, we would not have enough information to say much about their true talent.
rsynnott 6y ago
On a national level, the polls really did, well, not amazing, but fine, last time. 538's final polling average was Clinton+3.8, actual result was Clinton+2.1. That's not at all bad; you wouldn't call that an upset. The real problem was a lack of high-quality state-level polls, and in one or two cases major misses on what state polls there were; there wasn't all that much visibility on many states. And ultimately, the final result was so close that it was below the margin of error of any reasonable poll to detect. Even assuming the highest quality polls conducted in each state, every day, the best they'd have been able to say would have been, pretty much "It's 50/50", in retrospect.
- nl 6y ago
  And the state level polling errors were highly correlated, which 538 called out in their pre-election coverage, but people ignored.
- dragonwriter 6y ago
  > The real problem was a lack of high-quality state-level polls That wasn't even really a problem; the error in state polling was pretty consistent with the error in national polling; the problem with predictors other than 538 is they treated state variations from polling averages as independent, 538 correctly assessed them (based on past evidence) as highly correlated, which is why Trump had a nearly 1 in 3 chance in 538s forecast. The problem is people taking (honestly or just for the purpose of after-the-fact criticism) "1 in 3" to mean "absolutely won't happen".

verylittlemeat 6y ago

You're being downvoted but you're right. I'm starting to believe that sites like 538 feed off of the general public not understanding that a 20% chance to win is 1 in 5. Endless ink has been spilled about how the public does not understand statistics but very little effort has been made to communicate effectively. The whole thing seems like a navelgazing sideshow where the pollsters want to have it both ways. They want to claim their predictions are infallible but then when the public says they failed they want to backpedal with holier than thou "well, actually..." excuses.

aw1621107 6y ago
> Endless ink has been spilled about how the public does not understand statistics but very little effort has been made to communicate effectively. How, then, should statistics be appropriately communicated? > They want to claim their predictions are infallible but then when the public says they failed they want to backpedal with holier than thou "well, actually..." excuses. Do pollsters generally try to claim their numbers are infallible? If anything, the fact that margins of error are included in the results would seem to imply the opposite.
- verylittlemeat 6y ago
  I don't buy that they're trying to communicate in good faith. I remember very clearly sources like this* in 2016. There was so much hubris and minimizing of Trump that him having a 30% chance to win was like an irrelevant footnote to Hillary's coming coronation. *https://pbs.twimg.com/media/Cwc-j6YXUAEIMaM.jpg https://pbs.twimg.com/media/Cwc-j6YXUAEIMaM.jpg The pollsters take too much credit when they're right and deflect too much blame when they're wrong. They need to do a better job of being humble and stop trying to pass themselves off as apolitical number crunchers just giving us the facts.
  - aw1621107 6y ago
    > I don't buy that they're trying to communicate in good faith. What would "good faith" communication look like, then? > I remember very clearly sources like this* in 2016. Can you elaborate on what exactly is wrong with that source? (That isn't a pollster, too; that's a (meta?)analysis/aggregation, but that's a relatively minor nitpick) > They need to do a better job of being humble and stop trying to pass themselves off as apolitical number crunchers just giving us the facts. Are they themselves claiming that they are "giving us the facts", or is that how they are being represented by others? And are you talking about actual polls, or about predictions based on said polls?