5 ms·

He didn’t hedge at the end. Nate always writes the models before election season then doesn’t touch them apart from actual bug fixes. The model actually organic

He didn’t hedge at the end. Nate always writes the models before election season then doesn’t touch them apart from actual bug fixes. The model actually organically predicted 30%. I still think that’s about accurate. Maybe it should’ve been 40%. Everyone forgets that it was a pretty close election. Clinton could’ve won without the Comey announcement.

stinkbeetle 5mo ago
I think he did hedge (or "strategically bug fix"). The prediction for Trump went from IIRC around 15 to 30 in the last week or so. It was a big swing, IIRC with a lot of waffle around why it happened but not a lot of verifiable fact. > I still think that’s about accurate. Maybe it should’ve been 40%. It wasn't accurate. This is something people misunderstand about these predictions. If the 2016 election was held 100 times, Trump would have won 100 times. It's not the same as rolling dice. These election predictions don't say that. They say something like "the observations I have agree with scenarios that have Clinton winning, 70% of the time". Which is fine and correct as far as his data and model goes, but none of those scenarios were the reality he was trying to predict. They are all just figments of the model though. Getting down to the brass tacks, he predicted Clinton would win, and he was wrong. Which is fine, we just can't know anything about his process from that failure. Certainly we can't conclude that it was "accurate", since it was not. If we had a good sample of elections where he used the same process and built up a good record then sure.
- SilverBirch 5mo ago
  To give you a trivial example: The simplest way I can put this is that turn out varies based on the weather[1], and turn out is skewed by party. So if it rains on election day you are going to get a different result, and that result can flip the outcome of the election if the election is close. So it’s kind of a nonsense to say. “Trump would have won 100 times out of 100”. Are you saying Nate Silvers model should have had a perfect meteorological model to predict the weather? Or are you saying the election wasn’t close? In which case you’re just wrong on the facts. The 70% figure is saying “we know most of the information needed to determine what the outcome of the election will be but we don’t know everything so can’t be certain”. There is no process where you can know every factor that determines the result in advance with absolutely accuracy and I don’t know why people expect there would be. [1] https://www.sciencedirect.com/science/article/pii/S0261379422001299 https://www.sciencedirect.com/science/article/pii/S026137942...
  - stinkbeetle 5mo ago
    It's not nonsense. What's nonsense is to say Nate's prediction for the election was accurate or correct. It trivially was not. What it would be reasonable to say is if his model had correctly predicted the outcome of a significant sample of elections, then you could say his model has some accuracy or predictive power. But it still would never have been accurate or right in the specific instances it got wrong, that's just a misconception about how statistics and predictive models work. I hope this helps.
    - tekla 5mo ago
      It's so interesting to see how someone could so confidentially wrong and clearly show no knowledge of statistics.
    - SilverBirch 5mo ago
      What are you even classifying as accurate or correct? Do you take every 51% prediction from FiveThirtyEight and if the result is a win you consider that forecast accurate? And every 49% prediction must result in a loss? This just not how statistical forecasts work. >What it would be reasonable to say is if his model had correctly predicted the outcome of a significant sample of elections, then you could say his model has some accuracy or predictive power. I don't know why you're couching that in a hypothetical, FiveThirtyEight has repeatedly done that exercise. >But it still would never have been accurate or right in the specific instances it got wrong It is core to the concept of a probability that the result is going to go the opposite way from the prediction sometimes! It's meaningless to call it "wrong".
      - stinkbeetle 5mo ago
        > What are you even classifying as accurate or correct? When somebody gives a prediction of the outcome of an election? I classify it as correct if they predicted the candidate who wins. > Do you take every 51% prediction from FiveThirtyEight and if the result is a win you consider that forecast accurate? And every 49% prediction must result in a loss? This just not how statistical forecasts work. No, but it is the way to map statistical forecasts to reality. He was quite explicitly predicting the outcome of the actual election. That prediction was incorrect. The whole rating of the accuracy of these models is really snakeoil dressed up as science. There is a lot less rigorous science and a lot more feelings and adjusting numbers and twiddling formulas retrospectively than you were probably led to believe. Would a 99-1 for Trump model have been worse or less accurate than a 51-49 for Clinton model? Despite predicting the correct outcome whereas the Clinton model predicted the incorrect outcome? > I don't know why you're couching that in a hypothetical, FiveThirtyEight has repeatedly done that exercise. Not really with much rigor. Where are their reproducible published papers and data sets? They made their name with a bit of luck on a fairly predictable election, but were unable to show a significant advantage in their methods across a number of elections. > It is core to the concept of a probability that the result is going to go the opposite way from the prediction sometimes! It's meaningless to call it "wrong". No no, that's not true. There are two different things here. Firstly, if you had a model and method of predicting elections that you applied to a sample of elections and showed that it had a good ability to correctly predict, then you can say your model is a good prediction across typical elections. The model getting one wrong does not make it a bad model over a set of elections. It absolutely is wrong for that particular election though. And secondly if you use a model to make a prediction about a particular election, when your prediction turns out to be wrong, it was not retroactively correct because it just followed the model and you claim the model is good. That's just not how statistics or predictions work.
- birdland 5mo ago
  [dead]
- saberience 5mo ago
  That's where you're wrong, the election was very, very close. In fact, if roughly 40k voters (across three states) had switched from Trump to Hillary, she would have won, that's how close it was. 40k voters, that's really not very many. So it's hard to say whether Trump had a 30% chance of winning or 40% or whatever, but the election at most was a toss-up. Many random events could have resulted in a different outcome.
  - stinkbeetle 5mo ago
    You misunderstand my point. I am talking about the actual election that happened where these many random events that could have resulted in a different outcome did not happen. I was being a bit facetious maybe in my point. But the point is that the thing that is to be predicted is the actual real event that occurs in this universe. Silver made a prediction, and it was wrong. "Oh but it was only a 70% prediction" You can't 70% win an election. Silver's prediction was that Clinton would win, but he was not super confident about it. The prediction was wrong. He was right to not be super confident about it, but the prediction of who would win was still wrong.
    - zaphar 5mo ago
      Statistical likelihood is a measurement of the known data at the time. If you engage with the content otherwise then it's on you if you have the wrong takeaway. No one who makes a prediction based on a statistical model is going to be right every time. That doesn't mean it's not right to make a prediction. The statistical modeling can help you to be correct more often than not. And if you were going to be truly fair you would note that Nate in fact repeatedly said that it was still very much possible for Trump to win but that the current known polling data and other factors in his model pointed to a loss. 538's own post-mortem's on the event highlight that Trump was a very unusual candidate running in a very unusual election and as such the model was missing a lot of important information. They learned from the experience and adjusted the model going forward. Anyone complaining about that event is really just highlighting that they don't understand how statistical modeling works and are upset about how the model misled them or others which isn't Nate or 538's fault and is entirely on the consumer of their reporting. It's not like they didn't try to educate their consumers in their reporting.
- isityettime 5mo ago
  That's the beauty of this brand of pseudoscience. Statistical predictions of singular events like a particular election are totally unfalsifiable. You can just say "I guess we live in 30% world" or whatever, every time.
  - IAmBroom 5mo ago
    > Statistical predictions of singular events like a particular election are totally unfalsifiable. Yes. And the 2nd Law of Thermodynamics was just violated by millions of atoms within my lungs, that happened to increase in energy above the ambient average due to collisions. Clearly thermodynamics is pseudoscience, too!

stinkbeetle 5mo ago

I think he did hedge (or "strategically bug fix"). The prediction for Trump went from IIRC around 15 to 30 in the last week or so. It was a big swing, IIRC with a lot of waffle around why it happened but not a lot of verifiable fact. > I still think that’s about accurate. Maybe it should’ve been 40%. It wasn't accurate. This is something people misunderstand about these predictions. If the 2016 election was held 100 times, Trump would have won 100 times. It's not the same as rolling dice. These election predictions don't say that. They say something like "the observations I have agree with scenarios that have Clinton winning, 70% of the time". Which is fine and correct as far as his data and model goes, but none of those scenarios were the reality he was trying to predict. They are all just figments of the model though. Getting down to the brass tacks, he predicted Clinton would win, and he was wrong. Which is fine, we just can't know anything about his process from that failure. Certainly we can't conclude that it was "accurate", since it was not. If we had a good sample of elections where he used the same process and built up a good record then sure.

SilverBirch 5mo ago
To give you a trivial example: The simplest way I can put this is that turn out varies based on the weather[1], and turn out is skewed by party. So if it rains on election day you are going to get a different result, and that result can flip the outcome of the election if the election is close. So it’s kind of a nonsense to say. “Trump would have won 100 times out of 100”. Are you saying Nate Silvers model should have had a perfect meteorological model to predict the weather? Or are you saying the election wasn’t close? In which case you’re just wrong on the facts. The 70% figure is saying “we know most of the information needed to determine what the outcome of the election will be but we don’t know everything so can’t be certain”. There is no process where you can know every factor that determines the result in advance with absolutely accuracy and I don’t know why people expect there would be. [1] https://www.sciencedirect.com/science/article/pii/S0261379422001299 https://www.sciencedirect.com/science/article/pii/S026137942...
- stinkbeetle 5mo ago
  It's not nonsense. What's nonsense is to say Nate's prediction for the election was accurate or correct. It trivially was not. What it would be reasonable to say is if his model had correctly predicted the outcome of a significant sample of elections, then you could say his model has some accuracy or predictive power. But it still would never have been accurate or right in the specific instances it got wrong, that's just a misconception about how statistics and predictive models work. I hope this helps.
  - tekla 5mo ago
    It's so interesting to see how someone could so confidentially wrong and clearly show no knowledge of statistics.
  - SilverBirch 5mo ago
    What are you even classifying as accurate or correct? Do you take every 51% prediction from FiveThirtyEight and if the result is a win you consider that forecast accurate? And every 49% prediction must result in a loss? This just not how statistical forecasts work. >What it would be reasonable to say is if his model had correctly predicted the outcome of a significant sample of elections, then you could say his model has some accuracy or predictive power. I don't know why you're couching that in a hypothetical, FiveThirtyEight has repeatedly done that exercise. >But it still would never have been accurate or right in the specific instances it got wrong It is core to the concept of a probability that the result is going to go the opposite way from the prediction sometimes! It's meaningless to call it "wrong".
    - stinkbeetle 5mo ago
      > What are you even classifying as accurate or correct? When somebody gives a prediction of the outcome of an election? I classify it as correct if they predicted the candidate who wins. > Do you take every 51% prediction from FiveThirtyEight and if the result is a win you consider that forecast accurate? And every 49% prediction must result in a loss? This just not how statistical forecasts work. No, but it is the way to map statistical forecasts to reality. He was quite explicitly predicting the outcome of the actual election. That prediction was incorrect. The whole rating of the accuracy of these models is really snakeoil dressed up as science. There is a lot less rigorous science and a lot more feelings and adjusting numbers and twiddling formulas retrospectively than you were probably led to believe. Would a 99-1 for Trump model have been worse or less accurate than a 51-49 for Clinton model? Despite predicting the correct outcome whereas the Clinton model predicted the incorrect outcome? > I don't know why you're couching that in a hypothetical, FiveThirtyEight has repeatedly done that exercise. Not really with much rigor. Where are their reproducible published papers and data sets? They made their name with a bit of luck on a fairly predictable election, but were unable to show a significant advantage in their methods across a number of elections. > It is core to the concept of a probability that the result is going to go the opposite way from the prediction sometimes! It's meaningless to call it "wrong". No no, that's not true. There are two different things here. Firstly, if you had a model and method of predicting elections that you applied to a sample of elections and showed that it had a good ability to correctly predict, then you can say your model is a good prediction across typical elections. The model getting one wrong does not make it a bad model over a set of elections. It absolutely is wrong for that particular election though. And secondly if you use a model to make a prediction about a particular election, when your prediction turns out to be wrong, it was not retroactively correct because it just followed the model and you claim the model is good. That's just not how statistics or predictions work.
birdland 5mo ago
[dead]
saberience 5mo ago
That's where you're wrong, the election was very, very close. In fact, if roughly 40k voters (across three states) had switched from Trump to Hillary, she would have won, that's how close it was. 40k voters, that's really not very many. So it's hard to say whether Trump had a 30% chance of winning or 40% or whatever, but the election at most was a toss-up. Many random events could have resulted in a different outcome.
- stinkbeetle 5mo ago
  You misunderstand my point. I am talking about the actual election that happened where these many random events that could have resulted in a different outcome did not happen. I was being a bit facetious maybe in my point. But the point is that the thing that is to be predicted is the actual real event that occurs in this universe. Silver made a prediction, and it was wrong. "Oh but it was only a 70% prediction" You can't 70% win an election. Silver's prediction was that Clinton would win, but he was not super confident about it. The prediction was wrong. He was right to not be super confident about it, but the prediction of who would win was still wrong.
  - zaphar 5mo ago
    Statistical likelihood is a measurement of the known data at the time. If you engage with the content otherwise then it's on you if you have the wrong takeaway. No one who makes a prediction based on a statistical model is going to be right every time. That doesn't mean it's not right to make a prediction. The statistical modeling can help you to be correct more often than not. And if you were going to be truly fair you would note that Nate in fact repeatedly said that it was still very much possible for Trump to win but that the current known polling data and other factors in his model pointed to a loss. 538's own post-mortem's on the event highlight that Trump was a very unusual candidate running in a very unusual election and as such the model was missing a lot of important information. They learned from the experience and adjusted the model going forward. Anyone complaining about that event is really just highlighting that they don't understand how statistical modeling works and are upset about how the model misled them or others which isn't Nate or 538's fault and is entirely on the consumer of their reporting. It's not like they didn't try to educate their consumers in their reporting.
isityettime 5mo ago
That's the beauty of this brand of pseudoscience. Statistical predictions of singular events like a particular election are totally unfalsifiable. You can just say "I guess we live in 30% world" or whatever, every time.
- IAmBroom 5mo ago
  > Statistical predictions of singular events like a particular election are totally unfalsifiable. Yes. And the 2nd Law of Thermodynamics was just violated by millions of atoms within my lungs, that happened to increase in energy above the ambient average due to collisions. Clearly thermodynamics is pseudoscience, too!