8 ms·

FiveThirtyEight 2022 Election Forecast

21 points by cmoog 4y ago

egberts1 4y ago
New York Times that predicted a 93% chance of a Clinton landslide in 2016. And the same 538 concurred with 87% chance of a Clinton win. And the very same 538 that claimed a “Comey Effect” that purportedly botched their deep analytical effort, posthumously. https://fivethirtyeight.com/features/what-a-clinton-landslide-would-look-like/ https://fivethirtyeight.com/features/what-a-clinton-landslid...
- gizmo686 4y ago
  From the very first paragraph of your link. > Trump is a significant underdog — he has a 13 percent chance of winning the election according to our polls-only model and a 23 percent chance according to polls-plus. But those probabilities aren’t that small. For comparison, you have a 17 percent chance of losing a “game” of Russian roulette. A) Their more advanced model gave Trump a 23% chance, the 13% chance you mention is their simpler model. B) Things with a 23%, 13%, and 7% chance happen all the time C) You often have a better understanding of events after they happen relative to before. If you want to show a predictor is unreliable, you need to look at a bunch of their predictions and see how they perform in aggregate. While also considering how correlated the results are (and, if applicable, predicted to be)
  - egberts1 4y ago
    What more modeling does 538 need … to cover all their bases … and still miss the mark? As far as I have read of their blogs, they’re par for the course.
    - Sakos 4y ago
      Their predictions were far higher than anybody else. I'm not sure what you're expecting them to do. They can't just ignore data because it "feels" like it's more likely that Trump will win. Its not a number they arbitrarily decide on. Based on the numbers they had, they made the most accurate estimation they could.
      - egberts1 4y ago
        Have you actually gauged 538’s own probabilities against themselves? I mean it’s not like them attaining “The Founders” status (excuse me, Isaac Asimov), not even closed.
        gizmo686 4y ago
        You can view a bit of their analysis here: https://projects.fivethirtyeight.com/checking-our-work/presidential-elections/ https://projects.fivethirtyeight.com/checking-our-work/presi... Their seems to be a systemetic bias for them to be underconfident. That is to say, when they a result is likly (>50%) to happen, they underestimate how likely. Conversly, when they think an event is less likely (<50%), they overestimate its chances. Still, more likely events tended to occur more often than less likely events. Correcting for this systematic error would mean they should have given Clinton an even higher chance.
        ChrisLomont 4y ago
        >Have you actually gauged 538’s own probabilities against themselves? Yes, and the publish datasets of all their work to let others analyze them too. They're the most accurate of all the major sites. Here's their github site [1]. It contains tons of datasets, past performance, analysis tools, everything you want to actually check if your claims are true or false, and most importantly, how accurate 538 has been over time. Now try to do that analysis on, say, Fox News :) Good luck. [1] https://github.com/fivethirtyeight https://github.com/fivethirtyeight
    - Ma8ee 4y ago
      They didn’t miss the mark.
- cjohnson318 4y ago
  I followed the 2016 election and 538 very closely. Over the last week or so before the election was called, 538 gave Clinton a 3 out of 5 (or 60%) chance of winning, dramatically lower than any other outlet anywhere, and 35% lower than the NYT prediction.
- wbsss4412 4y ago
  Even with the benefit of hindsight, I don’t know how one can look back and say trump wasn’t an underdog going into the election. What exactly were they supposed to do? Ignore polls that show him behind and divine somehow that he was actually going to win based off of nothing?
  - Jonovono 4y ago
    Anyone paying attention was pretty certain Trump was going to win that. All you had to do was look at Trumps subreddit vs Hilarys. Look at Trumps rally turnouts. The energy behind Trump was insane and it seemed pretty clear something was going on in the media to make it sound like only a "minority" of people supported him. That's why I bet around $1000 with some coworkers he would win that were certain he was going to get destroyed. Note: This isn't me saying I support or don't support Trump.
    - francisofascii 4y ago
      So are you suggesting forecasters should factor in Reddit posts/votes and rally turnouts?
      - Jonovono 4y ago
        I don't care what they do.
    - elikoga 4y ago
      A minority did vote for trump though
      - meatsauce 4y ago
        President Trump won the popular vote in 30 of 50 states.
        rgifford 4y ago
        9 of those 30 states have more cows than people [1]. When your state is a single dude standing in a field filled with cows, getting the popular vote doesn't say much. 1. https://beef2live.com/story-cattle-inventory-vs-human-population-state-0-114255 https://beef2live.com/story-cattle-inventory-vs-human-popula...
- gorgonical 4y ago
  Echoing a sibling comment, it seems strange it's so common that people look at probabilities like this and say that because the thing that happened was less likely that the model was wrong. They say explicitly how this works: they run simulations and count winners. That means among their simulations is very likely to be an instance that is similar to the real outcome.
  - GameOfFrowns 4y ago
    >That means among their simulations is very likely to be an instance that is similar to the real outcome. So what? If you only make enough enough predictions then some of those will turn out predict the future correctly. It is how "The Simpsons did it first" meme came into being.
    - beardedetim 4y ago
      > If you only make enough enough predictions then some of those will turn out predict the future correctly. If we have a coin and we say "It's 50% chance it lands heads" and it lands tails, were we wrong? If we have a 20-sided die and we say "It has a 5% chance of landing on 20" and it lands on 20, were we wrong?
      - GameOfFrowns 4y ago
        Depends, is the coin a Democratic party establishment candidate and the predictor part of the liberal politico-media complex trying to prime an audience?
  - iconjack 4y ago
    Yes, I agree. If you are tasked for example with analyzing a coin to assess its bias, you could do a very thorough job, measuring precisely its mass distribution, its contours, etc and come up with an extremely accurate estimate that say, the probability of it landing Heads is 73.22%. If the coin is subsequently flipped and comes up Tails, it doesn't mean your analysis was wrong. On the other hand, 538's simulation was based on probabilities of winning each state, and as I think most will agree, before and after the election, the most important states were Wisconsin, Pennsylvania, and Michigan. On election day, 538 gave the following probabilities: Wisconsin 83.5% Clinton [1] Pennsylvania 77.0% Clinton [2] Michigan 78.9% Clinton [3] I think Nate really screwed the pooch here. The vast bulk of his effort should have been spent on these states. If that is in fact the case, then I retract my criticism. [1] https://projects.fivethirtyeight.com/2016-election-forecast/wisconsin/ https://projects.fivethirtyeight.com/2016-election-forecast/... [2] https://projects.fivethirtyeight.com/2016-election-forecast/pennsylvania/ https://projects.fivethirtyeight.com/2016-election-forecast/... [3] https://projects.fivethirtyeight.com/2016-election-forecast/michigan/ https://projects.fivethirtyeight.com/2016-election-forecast/...
- wly_cdgr 4y ago
  While it should have been obvious to anyone who had been to Ohio in the couple years before the election that Trump was the favorite, 538 did give Clinton a lower chance than other outlets. Also, 13% things happen all the time. Now if the model had said 0.13%, ok, maybe you side-eye it

21 points by cmoog 4y ago

New York Times that predicted a 93% chance of a Clinton landslide in 2016. And the same 538 concurred with 87% chance of a Clinton win. And the very same 538 that claimed a “Comey Effect” that purportedly botched their deep analytical effort, posthumously. https://fivethirtyeight.com/features/what-a-clinton-landslide-would-look-like/ https://fivethirtyeight.com/features/what-a-clinton-landslid...

gizmo686 4y ago
From the very first paragraph of your link. > Trump is a significant underdog — he has a 13 percent chance of winning the election according to our polls-only model and a 23 percent chance according to polls-plus. But those probabilities aren’t that small. For comparison, you have a 17 percent chance of losing a “game” of Russian roulette. A) Their more advanced model gave Trump a 23% chance, the 13% chance you mention is their simpler model. B) Things with a 23%, 13%, and 7% chance happen all the time C) You often have a better understanding of events after they happen relative to before. If you want to show a predictor is unreliable, you need to look at a bunch of their predictions and see how they perform in aggregate. While also considering how correlated the results are (and, if applicable, predicted to be)
- egberts1 4y ago
  What more modeling does 538 need … to cover all their bases … and still miss the mark? As far as I have read of their blogs, they’re par for the course.
  - Sakos 4y ago
    Their predictions were far higher than anybody else. I'm not sure what you're expecting them to do. They can't just ignore data because it "feels" like it's more likely that Trump will win. Its not a number they arbitrarily decide on. Based on the numbers they had, they made the most accurate estimation they could.
    - egberts1 4y ago
      Have you actually gauged 538’s own probabilities against themselves? I mean it’s not like them attaining “The Founders” status (excuse me, Isaac Asimov), not even closed.
      - gizmo686 4y ago
        You can view a bit of their analysis here: https://projects.fivethirtyeight.com/checking-our-work/presidential-elections/ https://projects.fivethirtyeight.com/checking-our-work/presi... Their seems to be a systemetic bias for them to be underconfident. That is to say, when they a result is likly (>50%) to happen, they underestimate how likely. Conversly, when they think an event is less likely (<50%), they overestimate its chances. Still, more likely events tended to occur more often than less likely events. Correcting for this systematic error would mean they should have given Clinton an even higher chance.
      - ChrisLomont 4y ago
        >Have you actually gauged 538’s own probabilities against themselves? Yes, and the publish datasets of all their work to let others analyze them too. They're the most accurate of all the major sites. Here's their github site [1]. It contains tons of datasets, past performance, analysis tools, everything you want to actually check if your claims are true or false, and most importantly, how accurate 538 has been over time. Now try to do that analysis on, say, Fox News :) Good luck. [1] https://github.com/fivethirtyeight https://github.com/fivethirtyeight
  - Ma8ee 4y ago
    They didn’t miss the mark.
cjohnson318 4y ago
I followed the 2016 election and 538 very closely. Over the last week or so before the election was called, 538 gave Clinton a 3 out of 5 (or 60%) chance of winning, dramatically lower than any other outlet anywhere, and 35% lower than the NYT prediction.
wbsss4412 4y ago
Even with the benefit of hindsight, I don’t know how one can look back and say trump wasn’t an underdog going into the election. What exactly were they supposed to do? Ignore polls that show him behind and divine somehow that he was actually going to win based off of nothing?
- Jonovono 4y ago
  Anyone paying attention was pretty certain Trump was going to win that. All you had to do was look at Trumps subreddit vs Hilarys. Look at Trumps rally turnouts. The energy behind Trump was insane and it seemed pretty clear something was going on in the media to make it sound like only a "minority" of people supported him. That's why I bet around $1000 with some coworkers he would win that were certain he was going to get destroyed. Note: This isn't me saying I support or don't support Trump.
  - francisofascii 4y ago
    So are you suggesting forecasters should factor in Reddit posts/votes and rally turnouts?
    - Jonovono 4y ago
      I don't care what they do.
  - elikoga 4y ago
    A minority did vote for trump though
    - meatsauce 4y ago
      President Trump won the popular vote in 30 of 50 states.
      - rgifford 4y ago
        9 of those 30 states have more cows than people [1]. When your state is a single dude standing in a field filled with cows, getting the popular vote doesn't say much. 1. https://beef2live.com/story-cattle-inventory-vs-human-population-state-0-114255 https://beef2live.com/story-cattle-inventory-vs-human-popula...
gorgonical 4y ago
Echoing a sibling comment, it seems strange it's so common that people look at probabilities like this and say that because the thing that happened was less likely that the model was wrong. They say explicitly how this works: they run simulations and count winners. That means among their simulations is very likely to be an instance that is similar to the real outcome.
- GameOfFrowns 4y ago
  >That means among their simulations is very likely to be an instance that is similar to the real outcome. So what? If you only make enough enough predictions then some of those will turn out predict the future correctly. It is how "The Simpsons did it first" meme came into being.
  - beardedetim 4y ago
    > If you only make enough enough predictions then some of those will turn out predict the future correctly. If we have a coin and we say "It's 50% chance it lands heads" and it lands tails, were we wrong? If we have a 20-sided die and we say "It has a 5% chance of landing on 20" and it lands on 20, were we wrong?
    - GameOfFrowns 4y ago
      Depends, is the coin a Democratic party establishment candidate and the predictor part of the liberal politico-media complex trying to prime an audience?
- iconjack 4y ago
  Yes, I agree. If you are tasked for example with analyzing a coin to assess its bias, you could do a very thorough job, measuring precisely its mass distribution, its contours, etc and come up with an extremely accurate estimate that say, the probability of it landing Heads is 73.22%. If the coin is subsequently flipped and comes up Tails, it doesn't mean your analysis was wrong. On the other hand, 538's simulation was based on probabilities of winning each state, and as I think most will agree, before and after the election, the most important states were Wisconsin, Pennsylvania, and Michigan. On election day, 538 gave the following probabilities: Wisconsin 83.5% Clinton [1] Pennsylvania 77.0% Clinton [2] Michigan 78.9% Clinton [3] I think Nate really screwed the pooch here. The vast bulk of his effort should have been spent on these states. If that is in fact the case, then I retract my criticism. [1] https://projects.fivethirtyeight.com/2016-election-forecast/wisconsin/ https://projects.fivethirtyeight.com/2016-election-forecast/... [2] https://projects.fivethirtyeight.com/2016-election-forecast/pennsylvania/ https://projects.fivethirtyeight.com/2016-election-forecast/... [3] https://projects.fivethirtyeight.com/2016-election-forecast/michigan/ https://projects.fivethirtyeight.com/2016-election-forecast/...
wly_cdgr 4y ago
While it should have been obvious to anyone who had been to Ohio in the couple years before the election that Trump was the favorite, 538 did give Clinton a lower chance than other outlets. Also, 13% things happen all the time. Now if the model had said 0.13%, ok, maybe you side-eye it