4 ms·

> I did too much bragging in the media and didn’t anticipate the extent to which public opinion toward FiveThirtyEight would shift once we became a corporate-ba

> I did too much bragging in the media and didn’t anticipate the extent to which public opinion toward FiveThirtyEight would shift once we became a corporate-backed incumbent rather than an eccentric upstart Can’t speak for everyone else, but it wasn’t this for me. It was about 2016 presidential that lost me. He tries to justify this later about how theirs was better than other outlets but I don’t care. Call it emotional, naive, unfair or whatever you want, but regardless I had zero interest in reading any of their predictions or analyses after that. Not even mad, just that to my experience they had one job and they didn’t fulfill it at the most important time. They went from appearing insightful to just one opinion amongst so many others.

shawabawa3 4mo ago
Because they said trump only had a 30% chance to win? What if they had said 49%? Would that have made their prediction worthless?
- bellowsgulch 4mo ago
  I mean everyone said he had a snowball’s chance in hell and then we ended up with him for two terms because the Democrats can’t stop fighting over the worse possible candidates to back that no one is asking for.
  - palata 4mo ago
    I find it interesting to blame it on the democrats. We ended up with him because enough people voted for him.
    - kjkjadksj 4mo ago
      The blame is put on democrats because when they lose its because they don’t turn out and when they win its because they do. It is quite simple really. Republicans are far more reliable voters. You can look at vote totals and see this pattern. Massive delta for democrats election over election and usually half that delta for republicans.
    - dwoldrich 4mo ago
      And he won the popular vote if you believe that all U.S. elections are secure and sacrosanct. He is diabolical at getting people to talk about him and think about him constantly. Joe Biden on the other hand was a senile wrecker for Build Back Better and the party finally made "the switch" to unelected Harris far too late in the process. Even if she was a great candidate, with her odd laughter and fascination with buses, there was not enough time to shape her candidacy. Her VP candidate choice was hobbled by rising anti-semitism in the party against Shapiro and perhaps concerns of being outshined by him. No, the Democrats did not do themselves any favors in the '24 election. Carter, Clinton and Obama were media creations, vaulting to national prominence out of nowhere. It helped that Clinton and Obama were great, charismatic choices. Now the traditional media is fragmented and weak. You're not seeing furtive vaulting attempts for potential phenoms like Newsome gain any traction. Who is the media going to be stuck with next time? Will it be take-two for Harris? WHEN, not if, Harris loses bigly to Vance, then the Democrats will absolutely be to blame. Where are their all new shiny, beautiful, erudite candidates that would need all four years to gestate and promote? Shouldn't we be getting acquainted with them now? I wager they're not going to appear, and we'll get more flunkies. My theory as to why is that those currently in power in the party do not share; they're aging out and hollowing out the party in the process. We're to the point now of collapse. I'm surprised a third party on the left hasn't yet formed.
      - palata 4mo ago
        Yeah I think the presidential election system may be the one to blame. Only two parties, both very much "to the right", doesn't feel like a sane system anymore. IMO the better way is to have the government represent the people, and representing the people generally means enough politicians and having to go for a consensus.
        dwoldrich 4mo ago
        Somehow "centrist" has taken on a negative connotation in Democratic circles, but I don't understand it. Far left politics are wrecker politics that have their moments and then (hopefully) get sent back to the political desert for a generation. Governing in a sustainable way usually means big tent politics with give and take on the small stuff. Bill Clinton epitomized that style.
    - xmcp123 4mo ago
      The Democrats reliably back the color "beige" as a candidate. Obama was different, and he won back to back. Biden succeeded, barely, because Trump was fresh in everyone's mind. But for some reason the Democrats have been allergic to charisma for far too long. Voting in the US, it feels like I am forced to choose between evil and incompetence.
      - macintux 4mo ago
        Your description sounds more like evil vs boring, which is an easy choice for me. And frankly I’ll vote for incompetence over evil too. Because, y’know, evil.
        amalcon 4mo ago
        Hell, when it comes to government, I easily prefer boring to not-boring in an executive all else being equal. Which it wasn't here, but still. I want the firebrands with big ideas to improve things in the legislature, and steady hands on the tiller in the executive. Unfortunately it seems that my compatriots want firebrands in both places.
        xmcp123 4mo ago
        I don’t think the choice is necessarily boring vs firebrands. Obama is a great example. He was not a firebrand, not boring, and was elected twice in a row. Charisma matters.
        amalcon 4mo ago
        Obama is a very unique case. I wouldn't exactly call him a firebrand, but his public persona managed to give the perception of being a steady hand on the tiller who also had big ideas. The only other presidential candidate in my lifetime to give a similar impression was Al Gore, but he leaned more into the "steady hand" thing to his detriment. He also, as you note, had far less charisma than Obama.
        4mo ago
    - jojobas 4mo ago
      Every politician is to blame for their losses, just as every politician own their wins. The people voted for Trump because the Dems failed to get the people to vote for them.
    - lmm 4mo ago
      Yes and no. It's a two-party system and a lot of people vote for candidate X to stop candidate Y getting in. Indeed IIRC a lot of democrat messaging around the previous election was explicitly about stopping Trump rather than the merits of whoever their guy was.
      - palata 4mo ago
        Enough people did not vote against Trump is my point. "The democrats" voted against Trump, otherwise they don't really count as part of "the democrats" for this election, right?
        lmm 4mo ago
        > Enough people did not vote against Trump is my point. The argument is that there were enough people who voted against Biden-Harris, or against the DNC process, to swing it.
  - doctorpangloss 4mo ago
    do you think nate silver is part of the problem or part of the solution? the turnout-of-demographic-groups-based election model is surely the underlying intelligence failure here.
  - ntonozzi 4mo ago
    Everyone said that except for 538. That's why 538 was worth reading.
  - coliveira 4mo ago
    Democrats will not let people choose candidates because that may be too dangerous for their interests. We'll never get good candidates as long as the current leadership is in control.
    - amanaplanacanal 4mo ago
      Do you not have primaries where you are?
      - oceanplexian 4mo ago
        What position was Kamala Harris in the DNC Primaries before she was appointed as the General Election Candidate? First place? Second Place? Surely she must have been in the top 3?
        amanaplanacanal 4mo ago
        Not sure what you think they should have done? There is no way to reorganize primaries that late in the year. It totally sucks that nobody tried to convince Biden not to run again before the primaries started though.
        adamors 4mo ago
        Plenty tried, old Joe thought he’ll clutch it out again. People were trying to dissuade him from running in 2023 already.
      - bombcar 4mo ago
        Denying that the democrats had two high-profile situations where the “wrong” candidate ended up running is denying the obvious. It’s manifest that Obama wasn’t to win the primary, and that the superdelegates exist for reasons.
- bachmeier 4mo ago
  When pressed before the election, Silver did not explain where Trump's much higher probability of winning came from. He predicted a Trump loss, Trump won, and he claimed victory because he gave Trump a better chance of winning. There's no way that strategy could have failed.
  - hungryhobbit 4mo ago
    Nate Silver is not a magician! He can't magically make polls reliable! All he (or anyone) can do is interpret or analyse poll results, and then surface their findings in a way a larger audience can understand. 538 did that better than any other poll analyst ... but they all got it wrong because the polls themselves were faulty. TLDR; You can't get water from a stone, and no one (not even Nate Silver) can get perfectly accurate predictions from (inherently flawed) polls!
    - deleted 4mo ago
      [deleted]
    - bachmeier 4mo ago
      > All he (or anyone) can do is interpret or analyse poll results, and then surface their findings in a way a larger audience can understand. He (or anybody) can make adjustments to the data. He was challenged to explain why his predictions were so different, but he wouldn't do it. > 538 did that better than any other poll analyst He made a binary prediction, and it was wrong. There's no such thing as "better" when you only have one outcome. Your prediction is either right or wrong. If by "better" you mean he was wrong but assigned a higher probability to a Trump victory, the best forecaster would have been someone that mechanically changed the probability of a Trump victory to slightly less than 50% no matter what the data said.
  - bonzini 4mo ago
    Silver claimed that his model was better because it predicted a high correlation between PA/MI/WI. A model that predicts a 30% chance of winning the election will be wrong 1 out of 3 times, which is not quite a coin flip but close enough.
fabian2k 4mo ago
FiveThirtyEight gave Trump a 30% chance. Their reporting did make clear that with margin was within the range of a normal polling error. And sometimes you get more than a normal polling error. It doesn't help that the US has a terrible election system that often leads to small margins in some states being decisive.
- kypro 4mo ago
  > FiveThirtyEight gave Trump a 30% chance. I know I'm being super conspiratorial here but why wouldn't all forecasters predict just between 30% - 70%? That way if they're "right" they can take the credit for it and if they're wrong they can say "well, we weren't that wrong". That's probably what I'd do anyway...
  - fabian2k 4mo ago
    Of course there's more than that. Predicting higher uncertainty than warranted would be a different failure in the model. But that didn't really happen in that election.
  - volkl48 4mo ago
    It implies a close race or a strong reason to believe there's some sort of systemic polling miss, and if it's a blowout you still look pretty bad. Especially if you don't have some kind of good explanation for the miss/you keep making those kinds of misses frequently. Also there's more going in those forecasts besides just the "% chance to win". There's expected results in terms of %'s of the vote for the candidates, and that's what people tend to focus on for actually analyzing your performance and credibility after the fact. You getting the outcome correct but being off by 20 points on the margin is a much worse performance than you getting the outcome wrong but being within 0.5 points of the margin. (ex: Results are 49.75/50.25, you predicted 30/70, another outlet predicted 50.25/49.75).
  - coliveira 4mo ago
    You're completely right, although I believe this number was not decided personally. They just happened to pick algorithms that have this "nice" property because it will lead to the same result.
rurp 4mo ago
I had complaints about 538, especially the early days, but don't understand this critique at all. A 30% chance hitting is completely unremarkable, and it was a perfectly reasonable reading of the evidence at the time. Nate isn't wrong that conventional wisdom was way off, with even supposedly statistical models giving Hillary a 99% chance of winning. Elections, like many things, have some inherent uncertainty. A several point polling error is normal, so a candidate who is down a couple points on election day has a decent shot of winning.
- Retric 4mo ago
  Not just that, predictions also impact voter participation.
- munchler 4mo ago
  I think this is all true, but it dodges the bigger issue. A presidential election has a binary outcome: yes/no, win/lose. If your statistical model doesn’t contain this single bit in its output, then it doesn’t meet the minimum requirement for being a prediction. Now you might say that it was on me as a consumer to understand this in 2016, but I remember the look of total shock on Nate Silver’s face when he called the winner on live TV that night, so clearly he didn’t really understand it either. Lesson learned for all of us, I guess.
  - causalmodels 4mo ago
    > If your statistical model doesn’t contain this single bit in its output, then it doesn’t meet the minimum requirement for being a prediction. I would be fascinated to know how you model a coin flip.
    - munchler 4mo ago
      Well, that’s my point. You can model a coin flip, but you can’t predict it. A model and a prediction are two different things.
- lacewing 4mo ago
legitster 4mo ago
If I say you have a 50% chance to win a coin flip and you lose it, that doesn't mean I'm wrong. A key thing though is 538 did regularly test the calibration their models: https://web.archive.org/web/20190410030104/https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ https://web.archive.org/web/20190410030104/https://fivethirt... > "What you’ll find, though, is that our calibration has generally been very, very good. For instance, out of the 5,589 events (between sports and politics combined) that we said had a 70 chance of happening (rounded to the nearest 5 percent), they in fact occurred 71 percent of the time. Or of the 55,853 events that we said had about a 5 percent chance of occurring, they happened 4 percent of the time."
tekla 4mo ago
> Call it emotional, naive, unfair or whatever you want Yep definitely all those. Why is it so hard to admit 30% is not 0%?
jackmott42 4mo ago
Nate was a huge outlier in that prediction, he gave trump a better chance than almost anyone else that I can recall, so why are you mad at him about that? What made me mad is Nate seemed to turn into a MAGA troll himself after that election.
jp57 4mo ago
I really think a majority of NYTimes and ABCnews consumers don't know the difference between a 2/3 chance (super close) of winning and 2/3 of the vote (a landslide).
- Traster 4mo ago
  Apparently neither do a big chunk of HN readers.
tombert 4mo ago
A dice roll has a 16.6% chance of landing on any given side, meaning an 83% chance of not landing on that side. If you guessed a "two", and it landed on "two, I wouldn't really be that impressed, even though there was an 83% probability going against you.
bryanlarsen 4mo ago
538 used the example of Trump having approximately the same chance of winning the 2016 presidential election as the Cavaliers had of winning the NBA championship round vs the Warriors. Both Trump and the Cavaliers won with a ~25% predicted chance. 538 made very clear with this analogy that both Trump and the Cavs were underdogs, and that both had a solid chance of winning.
afavour 4mo ago
I think this gets to the core of why a lot of this election prediction stuff doesn't work. People just don't parse the numbers the way the authors intend. FiveThirtyEight had Trump at a 30% chance of winning, and he won. The model wasn't wrong. The less likely of two outcomes occurred. Even if they'd had him at 1% they still wouldn't technically have been wrong though I think complaints might be more warranted. If they had Trump at 49% would you have still been angry? What about at 51%? Would it have been okay then?
- coliveira 4mo ago
  Technically this is right. But if that is the case (and it seems to be), then a coin flip is better than their models. Because we only care about the current election, not a sequence of 1000 elections (which will not happen, by the way).
  - afavour 4mo ago
    I don’t disagree, I think the tea leaf reading is ultimately pretty futile. But at the same time I do think it’s valid to say it’s more than a coin flip. The polling data over the election cycle showed that Trump had a smaller but still legitimate chance of winning. The data was different in 2020, when he lost.
  - dogleash 4mo ago
    > But if that is the case (and it seems to be), then a coin flip is better than their models. If a coin flip is the necessary mental model to remind you both things can happen, then sure. People just love horse race coverage. Silver gave us the most accurate horse race coverage. Maybe the lesson is stop following horse race coverage. But most people went back to the tea leave readers. That way when the election was over, it can justifiably be the charlatan's fault that viewers got over-invested in their predictive capabilities.
  - reed1234 4mo ago
    So use a coin flip to predict the weather then
softwaredoug 4mo ago
We should have a drinking game in Nate Silver thread anyone complains about 2016 prediction. Then everyone piles on to point out how probabilities work.
- ninth_ant 4mo ago
  It wasn't even a complaint, just a personal anecdote to help share some context as to why the site may have failed to retain consumer interest post-2016. But yes I'll join you with the liver damage and drink 17 shots.
- MostlyStable 4mo ago
  I long ago stopped commenting on threads (in /r/CFB) about the SP+ college football model because it go so, so, so tiring correcting the exact same misunderstandings about things as fundamental as what the model was even purporting to do let alone, how it worked. It is incredibly disheartening to me to see the staggering level of statistical illiteracy on display in any online conversation about a statistical model. I've come to the conclusion that it's not worth my, or any one else's time. I'm glad these things exist, and the communities who want them will self select. It's not use trying to convert haters.
  - rurp 4mo ago
    I've long resigned myself to the fact that a lay audience will usually struggle to understand basic statistics. On HN though I don't expect to see those same issues much but have been surprised to see how many posters have trouble differentiating between 30%, 50%, and 99% probabilities.
reed1234 4mo ago
That’s like saying “there was only a 30 percent chance of rain today and it rained, so I will never look at the weather forecast again.”
topaz0 4mo ago
As a younger man I would have been with the commenters mansplaining probability, but I've aged into realizing that thinking of the election like a marble pulled from an urn whose contents we have probed with polling is just as bad as thinking of it as deterministic. The reason people read fivethirtyeight, probability-savvy or not, was almost entirely to be told what was going to happen, which is sort of incompatible with feeling you can do anything about it. In that way it's probably worse than old-fashioned pundit-driven horse race coverage because it has an air of scientific authority.
baubino 4mo ago
Huh. I first started listening to 538 in the run-up to the 2016 election and started really paying attention to them precisely because their 30% figure was so much higher than all the other polls. It was shocking to me then (and still is now reading your comment) that people didn’t seem to understand that 30% in the context of that particular election and that particular candidate suggested a remarkably high chance of winning, not a really low chance of winning. It’s a strange thing where people seem to think that less than 50% = not happening.
albedoa 4mo ago
> Call it emotional, naive, unfair or whatever you want We would need a pass from the mods lol.
cm2012 4mo ago
If you use 538 data, on average you make money betting. Its more correct than not
NeutralCrane 4mo ago
Hard to believe that even 10 years later, people who don’t understand probability are still hung up on an event with a 30% chance of happening, happening.

shawabawa3 4mo ago

Because they said trump only had a 30% chance to win? What if they had said 49%? Would that have made their prediction worthless?

bellowsgulch 4mo ago
I mean everyone said he had a snowball’s chance in hell and then we ended up with him for two terms because the Democrats can’t stop fighting over the worse possible candidates to back that no one is asking for.
- palata 4mo ago
  I find it interesting to blame it on the democrats. We ended up with him because enough people voted for him.
  - kjkjadksj 4mo ago
    The blame is put on democrats because when they lose its because they don’t turn out and when they win its because they do. It is quite simple really. Republicans are far more reliable voters. You can look at vote totals and see this pattern. Massive delta for democrats election over election and usually half that delta for republicans.
  - dwoldrich 4mo ago
    And he won the popular vote if you believe that all U.S. elections are secure and sacrosanct. He is diabolical at getting people to talk about him and think about him constantly. Joe Biden on the other hand was a senile wrecker for Build Back Better and the party finally made "the switch" to unelected Harris far too late in the process. Even if she was a great candidate, with her odd laughter and fascination with buses, there was not enough time to shape her candidacy. Her VP candidate choice was hobbled by rising anti-semitism in the party against Shapiro and perhaps concerns of being outshined by him. No, the Democrats did not do themselves any favors in the '24 election. Carter, Clinton and Obama were media creations, vaulting to national prominence out of nowhere. It helped that Clinton and Obama were great, charismatic choices. Now the traditional media is fragmented and weak. You're not seeing furtive vaulting attempts for potential phenoms like Newsome gain any traction. Who is the media going to be stuck with next time? Will it be take-two for Harris? WHEN, not if, Harris loses bigly to Vance, then the Democrats will absolutely be to blame. Where are their all new shiny, beautiful, erudite candidates that would need all four years to gestate and promote? Shouldn't we be getting acquainted with them now? I wager they're not going to appear, and we'll get more flunkies. My theory as to why is that those currently in power in the party do not share; they're aging out and hollowing out the party in the process. We're to the point now of collapse. I'm surprised a third party on the left hasn't yet formed.
    - palata 4mo ago
      Yeah I think the presidential election system may be the one to blame. Only two parties, both very much "to the right", doesn't feel like a sane system anymore. IMO the better way is to have the government represent the people, and representing the people generally means enough politicians and having to go for a consensus.
      - dwoldrich 4mo ago
        Somehow "centrist" has taken on a negative connotation in Democratic circles, but I don't understand it. Far left politics are wrecker politics that have their moments and then (hopefully) get sent back to the political desert for a generation. Governing in a sustainable way usually means big tent politics with give and take on the small stuff. Bill Clinton epitomized that style.
  - xmcp123 4mo ago
    The Democrats reliably back the color "beige" as a candidate. Obama was different, and he won back to back. Biden succeeded, barely, because Trump was fresh in everyone's mind. But for some reason the Democrats have been allergic to charisma for far too long. Voting in the US, it feels like I am forced to choose between evil and incompetence.
    - macintux 4mo ago
      Your description sounds more like evil vs boring, which is an easy choice for me. And frankly I’ll vote for incompetence over evil too. Because, y’know, evil.
      - amalcon 4mo ago
        Hell, when it comes to government, I easily prefer boring to not-boring in an executive all else being equal. Which it wasn't here, but still. I want the firebrands with big ideas to improve things in the legislature, and steady hands on the tiller in the executive. Unfortunately it seems that my compatriots want firebrands in both places.
        xmcp123 4mo ago
        I don’t think the choice is necessarily boring vs firebrands. Obama is a great example. He was not a firebrand, not boring, and was elected twice in a row. Charisma matters.
        amalcon 4mo ago
        Obama is a very unique case. I wouldn't exactly call him a firebrand, but his public persona managed to give the perception of being a steady hand on the tiller who also had big ideas. The only other presidential candidate in my lifetime to give a similar impression was Al Gore, but he leaned more into the "steady hand" thing to his detriment. He also, as you note, had far less charisma than Obama.
        4mo ago
  - jojobas 4mo ago
    Every politician is to blame for their losses, just as every politician own their wins. The people voted for Trump because the Dems failed to get the people to vote for them.
  - lmm 4mo ago
    Yes and no. It's a two-party system and a lot of people vote for candidate X to stop candidate Y getting in. Indeed IIRC a lot of democrat messaging around the previous election was explicitly about stopping Trump rather than the merits of whoever their guy was.
    - palata 4mo ago
      Enough people did not vote against Trump is my point. "The democrats" voted against Trump, otherwise they don't really count as part of "the democrats" for this election, right?
      - lmm 4mo ago
        > Enough people did not vote against Trump is my point. The argument is that there were enough people who voted against Biden-Harris, or against the DNC process, to swing it.
- doctorpangloss 4mo ago
  do you think nate silver is part of the problem or part of the solution? the turnout-of-demographic-groups-based election model is surely the underlying intelligence failure here.
- ntonozzi 4mo ago
  Everyone said that except for 538. That's why 538 was worth reading.
- coliveira 4mo ago
  Democrats will not let people choose candidates because that may be too dangerous for their interests. We'll never get good candidates as long as the current leadership is in control.
  - amanaplanacanal 4mo ago
    Do you not have primaries where you are?
    - oceanplexian 4mo ago
      What position was Kamala Harris in the DNC Primaries before she was appointed as the General Election Candidate? First place? Second Place? Surely she must have been in the top 3?
      - amanaplanacanal 4mo ago
        Not sure what you think they should have done? There is no way to reorganize primaries that late in the year. It totally sucks that nobody tried to convince Biden not to run again before the primaries started though.
        adamors 4mo ago
        Plenty tried, old Joe thought he’ll clutch it out again. People were trying to dissuade him from running in 2023 already.
    - bombcar 4mo ago
      Denying that the democrats had two high-profile situations where the “wrong” candidate ended up running is denying the obvious. It’s manifest that Obama wasn’t to win the primary, and that the superdelegates exist for reasons.
bachmeier 4mo ago
When pressed before the election, Silver did not explain where Trump's much higher probability of winning came from. He predicted a Trump loss, Trump won, and he claimed victory because he gave Trump a better chance of winning. There's no way that strategy could have failed.
- hungryhobbit 4mo ago
  Nate Silver is not a magician! He can't magically make polls reliable! All he (or anyone) can do is interpret or analyse poll results, and then surface their findings in a way a larger audience can understand. 538 did that better than any other poll analyst ... but they all got it wrong because the polls themselves were faulty. TLDR; You can't get water from a stone, and no one (not even Nate Silver) can get perfectly accurate predictions from (inherently flawed) polls!
  - deleted 4mo ago
    [deleted]
  - bachmeier 4mo ago
    > All he (or anyone) can do is interpret or analyse poll results, and then surface their findings in a way a larger audience can understand. He (or anybody) can make adjustments to the data. He was challenged to explain why his predictions were so different, but he wouldn't do it. > 538 did that better than any other poll analyst He made a binary prediction, and it was wrong. There's no such thing as "better" when you only have one outcome. Your prediction is either right or wrong. If by "better" you mean he was wrong but assigned a higher probability to a Trump victory, the best forecaster would have been someone that mechanically changed the probability of a Trump victory to slightly less than 50% no matter what the data said.
- bonzini 4mo ago
  Silver claimed that his model was better because it predicted a high correlation between PA/MI/WI. A model that predicts a 30% chance of winning the election will be wrong 1 out of 3 times, which is not quite a coin flip but close enough.

fabian2k 4mo ago

FiveThirtyEight gave Trump a 30% chance. Their reporting did make clear that with margin was within the range of a normal polling error. And sometimes you get more than a normal polling error. It doesn't help that the US has a terrible election system that often leads to small margins in some states being decisive.

kypro 4mo ago
> FiveThirtyEight gave Trump a 30% chance. I know I'm being super conspiratorial here but why wouldn't all forecasters predict just between 30% - 70%? That way if they're "right" they can take the credit for it and if they're wrong they can say "well, we weren't that wrong". That's probably what I'd do anyway...
- fabian2k 4mo ago
  Of course there's more than that. Predicting higher uncertainty than warranted would be a different failure in the model. But that didn't really happen in that election.
- volkl48 4mo ago
  It implies a close race or a strong reason to believe there's some sort of systemic polling miss, and if it's a blowout you still look pretty bad. Especially if you don't have some kind of good explanation for the miss/you keep making those kinds of misses frequently. Also there's more going in those forecasts besides just the "% chance to win". There's expected results in terms of %'s of the vote for the candidates, and that's what people tend to focus on for actually analyzing your performance and credibility after the fact. You getting the outcome correct but being off by 20 points on the margin is a much worse performance than you getting the outcome wrong but being within 0.5 points of the margin. (ex: Results are 49.75/50.25, you predicted 30/70, another outlet predicted 50.25/49.75).
- coliveira 4mo ago
  You're completely right, although I believe this number was not decided personally. They just happened to pick algorithms that have this "nice" property because it will lead to the same result.

rurp 4mo ago

I had complaints about 538, especially the early days, but don't understand this critique at all. A 30% chance hitting is completely unremarkable, and it was a perfectly reasonable reading of the evidence at the time. Nate isn't wrong that conventional wisdom was way off, with even supposedly statistical models giving Hillary a 99% chance of winning. Elections, like many things, have some inherent uncertainty. A several point polling error is normal, so a candidate who is down a couple points on election day has a decent shot of winning.

Retric 4mo ago
Not just that, predictions also impact voter participation.
munchler 4mo ago
I think this is all true, but it dodges the bigger issue. A presidential election has a binary outcome: yes/no, win/lose. If your statistical model doesn’t contain this single bit in its output, then it doesn’t meet the minimum requirement for being a prediction. Now you might say that it was on me as a consumer to understand this in 2016, but I remember the look of total shock on Nate Silver’s face when he called the winner on live TV that night, so clearly he didn’t really understand it either. Lesson learned for all of us, I guess.
- causalmodels 4mo ago
  > If your statistical model doesn’t contain this single bit in its output, then it doesn’t meet the minimum requirement for being a prediction. I would be fascinated to know how you model a coin flip.
  - munchler 4mo ago
    Well, that’s my point. You can model a coin flip, but you can’t predict it. A model and a prediction are two different things.
lacewing 4mo ago

legitster 4mo ago

If I say you have a 50% chance to win a coin flip and you lose it, that doesn't mean I'm wrong. A key thing though is 538 did regularly test the calibration their models: https://web.archive.org/web/20190410030104/https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ https://web.archive.org/web/20190410030104/https://fivethirt... > "What you’ll find, though, is that our calibration has generally been very, very good. For instance, out of the 5,589 events (between sports and politics combined) that we said had a 70 chance of happening (rounded to the nearest 5 percent), they in fact occurred 71 percent of the time. Or of the 55,853 events that we said had about a 5 percent chance of occurring, they happened 4 percent of the time."

tekla 4mo ago

> Call it emotional, naive, unfair or whatever you want Yep definitely all those. Why is it so hard to admit 30% is not 0%?

jackmott42 4mo ago

Nate was a huge outlier in that prediction, he gave trump a better chance than almost anyone else that I can recall, so why are you mad at him about that? What made me mad is Nate seemed to turn into a MAGA troll himself after that election.

jp57 4mo ago

I really think a majority of NYTimes and ABCnews consumers don't know the difference between a 2/3 chance (super close) of winning and 2/3 of the vote (a landslide).

Traster 4mo ago
Apparently neither do a big chunk of HN readers.

tombert 4mo ago

A dice roll has a 16.6% chance of landing on any given side, meaning an 83% chance of not landing on that side. If you guessed a "two", and it landed on "two, I wouldn't really be that impressed, even though there was an 83% probability going against you.

bryanlarsen 4mo ago

538 used the example of Trump having approximately the same chance of winning the 2016 presidential election as the Cavaliers had of winning the NBA championship round vs the Warriors. Both Trump and the Cavaliers won with a ~25% predicted chance. 538 made very clear with this analogy that both Trump and the Cavs were underdogs, and that both had a solid chance of winning.

afavour 4mo ago

I think this gets to the core of why a lot of this election prediction stuff doesn't work. People just don't parse the numbers the way the authors intend. FiveThirtyEight had Trump at a 30% chance of winning, and he won. The model wasn't wrong. The less likely of two outcomes occurred. Even if they'd had him at 1% they still wouldn't technically have been wrong though I think complaints might be more warranted. If they had Trump at 49% would you have still been angry? What about at 51%? Would it have been okay then?

coliveira 4mo ago
Technically this is right. But if that is the case (and it seems to be), then a coin flip is better than their models. Because we only care about the current election, not a sequence of 1000 elections (which will not happen, by the way).
- afavour 4mo ago
  I don’t disagree, I think the tea leaf reading is ultimately pretty futile. But at the same time I do think it’s valid to say it’s more than a coin flip. The polling data over the election cycle showed that Trump had a smaller but still legitimate chance of winning. The data was different in 2020, when he lost.
- dogleash 4mo ago
  > But if that is the case (and it seems to be), then a coin flip is better than their models. If a coin flip is the necessary mental model to remind you both things can happen, then sure. People just love horse race coverage. Silver gave us the most accurate horse race coverage. Maybe the lesson is stop following horse race coverage. But most people went back to the tea leave readers. That way when the election was over, it can justifiably be the charlatan's fault that viewers got over-invested in their predictive capabilities.
- reed1234 4mo ago
  So use a coin flip to predict the weather then

softwaredoug 4mo ago

We should have a drinking game in Nate Silver thread anyone complains about 2016 prediction. Then everyone piles on to point out how probabilities work.

ninth_ant 4mo ago
It wasn't even a complaint, just a personal anecdote to help share some context as to why the site may have failed to retain consumer interest post-2016. But yes I'll join you with the liver damage and drink 17 shots.
MostlyStable 4mo ago
I long ago stopped commenting on threads (in /r/CFB) about the SP+ college football model because it go so, so, so tiring correcting the exact same misunderstandings about things as fundamental as what the model was even purporting to do let alone, how it worked. It is incredibly disheartening to me to see the staggering level of statistical illiteracy on display in any online conversation about a statistical model. I've come to the conclusion that it's not worth my, or any one else's time. I'm glad these things exist, and the communities who want them will self select. It's not use trying to convert haters.
- rurp 4mo ago
  I've long resigned myself to the fact that a lay audience will usually struggle to understand basic statistics. On HN though I don't expect to see those same issues much but have been surprised to see how many posters have trouble differentiating between 30%, 50%, and 99% probabilities.

reed1234 4mo ago

That’s like saying “there was only a 30 percent chance of rain today and it rained, so I will never look at the weather forecast again.”

topaz0 4mo ago

As a younger man I would have been with the commenters mansplaining probability, but I've aged into realizing that thinking of the election like a marble pulled from an urn whose contents we have probed with polling is just as bad as thinking of it as deterministic. The reason people read fivethirtyeight, probability-savvy or not, was almost entirely to be told what was going to happen, which is sort of incompatible with feeling you can do anything about it. In that way it's probably worse than old-fashioned pundit-driven horse race coverage because it has an air of scientific authority.

baubino 4mo ago

Huh. I first started listening to 538 in the run-up to the 2016 election and started really paying attention to them precisely because their 30% figure was so much higher than all the other polls. It was shocking to me then (and still is now reading your comment) that people didn’t seem to understand that 30% in the context of that particular election and that particular candidate suggested a remarkably high chance of winning, not a really low chance of winning. It’s a strange thing where people seem to think that less than 50% = not happening.

albedoa 4mo ago

> Call it emotional, naive, unfair or whatever you want We would need a pass from the mods lol.

cm2012 4mo ago

If you use 538 data, on average you make money betting. Its more correct than not

NeutralCrane 4mo ago

Hard to believe that even 10 years later, people who don’t understand probability are still hung up on an event with a 30% chance of happening, happening.