5 ms·

Agree that we should be thoughtful of the misses as well as the hits, and he's had some misses. However, I didn't recall Silver getting the special MA Senate e

by aamar 16y ago

Agree that we should be thoughtful of the misses as well as the hits, and he's had some misses. However, I didn't recall Silver getting the special MA Senate election of Scott Brown wrong, and he seems not to have: (http://www.fivethirtyeight.com/2010/01/538-model-posits-brown-as-31-favorite.html http://www.fivethirtyeight.com/2010/01/538-model-posits-brow...). It would have been surprising if he did, since it was an easy one. Were you thinking of a different election?

spec 16y ago
You're right - it appears that he updated his model right before the elections and I only saw his earlier posts. However, he had a post in 2009 entitled "Could a Republican Win Ted Kennedy's Senate Seat?" with his answer being "Very probably not."
- wtallis 16y ago
  His modus operandi is to build a model and feed it data as it becomes available. So when polling of a race is inconsistent and infrequent until right before the election, you shouldn't imply that it is at all improper behavior or Nate Silver's fault if his model and thus his prediction change drastically right before an election. As for the early, subjective post about the odds, that was posted on 27 August 2009, just days after Ted Kennedy died, long before there was any concrete information about who would run even for the primaries, and before there was any concrete information about the electoral influence of the tea party movement. And Nate Silver was still probably right that Romney wouldn't have been able to take Kennedy's seat, since Romney wouldn't have attracted as many of the Tea Party voters.
  - spec 16y ago
    Based on your post, it appears that you don't understand how things like statistical significance work. If there is not enough data to make a prediction, you simply should not make a prediction and wait. If you ignore this and your whole business is based on the accuracy of your predictions, expect to be held accountable. You also apparently didn't read his post I mentioned. It does not talk about just Romney. Here is the direct quote: "As for other Republicans in Massachusetts, their prospects don't figure to be much better." Silver's whole failed business was based on what Taleb called the "Ludic fallacy" in The Black Swan.
    - philwelch 16y ago
      "Based on your post, it appears that you don't understand how things like statistical significance work. If there is not enough data to make a prediction, you simply should not make a prediction and wait." Silver doesn't make "predictions", he gives odds. Giving odds implicitly admits that there exists a chance of beating those odds, and attempts to quantify how big or small that chance is. Sure, improbable things happen. That does not make them not improbable. Really, if someone is giving probabilities, you don't just count how many times the lower-probability events happened against him, you take all of his 20% probabilities and look to see whether about 20% of them ended up happening. Look at it the other way--if all of Nate Silver's "predictions" came true, he'd be way more wrong than if some of them didn't come true, because he's not predicting things with 100% probability! Of all the things he thinks are 70% likely to happen, he's just as wrong if 100% of them happen as if 40% of them happen--he's only right if 70% of his 70% shots happen!
      - spec 16y ago
        You are absolutely and factually wrong. He absolutely calls them predictions when he is right. Here is a direct quote from Silver: "predicting 49 of 50 states correctly in the presidential election." http://fivethirtyeight.blogs.nytimes.com/2010/08/25/welcome-and-welcome-back/ http://fivethirtyeight.blogs.nytimes.com/2010/08/25/welcome-...
        philwelch 16y ago
        OK--he does, in fact, use the word "prediction", but on the same token, he's still fairly open about his predictions being probabilistic in nature, and about the predictions changing over time as more evidence is presented. But you're going to have to do better than pounce on my word choice if you want to prove your point.
- philwelch 16y ago
  The facts of the matter at that time were different from the facts of the matter when Scott Brown eventually won. When you come out and say it's only 20% probable for a given outcome to happen, and that outcome happens, it doesn't mean you were wrong, it just means a long shot happened. In fact, you would expect just about exactly 1 out of every 5 20% longshots to end up happening.

by aamar 16y ago

spec 16y ago

You're right - it appears that he updated his model right before the elections and I only saw his earlier posts. However, he had a post in 2009 entitled "Could a Republican Win Ted Kennedy's Senate Seat?" with his answer being "Very probably not."

wtallis 16y ago
His modus operandi is to build a model and feed it data as it becomes available. So when polling of a race is inconsistent and infrequent until right before the election, you shouldn't imply that it is at all improper behavior or Nate Silver's fault if his model and thus his prediction change drastically right before an election. As for the early, subjective post about the odds, that was posted on 27 August 2009, just days after Ted Kennedy died, long before there was any concrete information about who would run even for the primaries, and before there was any concrete information about the electoral influence of the tea party movement. And Nate Silver was still probably right that Romney wouldn't have been able to take Kennedy's seat, since Romney wouldn't have attracted as many of the Tea Party voters.
- spec 16y ago
  Based on your post, it appears that you don't understand how things like statistical significance work. If there is not enough data to make a prediction, you simply should not make a prediction and wait. If you ignore this and your whole business is based on the accuracy of your predictions, expect to be held accountable. You also apparently didn't read his post I mentioned. It does not talk about just Romney. Here is the direct quote: "As for other Republicans in Massachusetts, their prospects don't figure to be much better." Silver's whole failed business was based on what Taleb called the "Ludic fallacy" in The Black Swan.
  - philwelch 16y ago
    "Based on your post, it appears that you don't understand how things like statistical significance work. If there is not enough data to make a prediction, you simply should not make a prediction and wait." Silver doesn't make "predictions", he gives odds. Giving odds implicitly admits that there exists a chance of beating those odds, and attempts to quantify how big or small that chance is. Sure, improbable things happen. That does not make them not improbable. Really, if someone is giving probabilities, you don't just count how many times the lower-probability events happened against him, you take all of his 20% probabilities and look to see whether about 20% of them ended up happening. Look at it the other way--if all of Nate Silver's "predictions" came true, he'd be way more wrong than if some of them didn't come true, because he's not predicting things with 100% probability! Of all the things he thinks are 70% likely to happen, he's just as wrong if 100% of them happen as if 40% of them happen--he's only right if 70% of his 70% shots happen!
    - spec 16y ago
      You are absolutely and factually wrong. He absolutely calls them predictions when he is right. Here is a direct quote from Silver: "predicting 49 of 50 states correctly in the presidential election." http://fivethirtyeight.blogs.nytimes.com/2010/08/25/welcome-and-welcome-back/ http://fivethirtyeight.blogs.nytimes.com/2010/08/25/welcome-...
      - philwelch 16y ago
        OK--he does, in fact, use the word "prediction", but on the same token, he's still fairly open about his predictions being probabilistic in nature, and about the predictions changing over time as more evidence is presented. But you're going to have to do better than pounce on my word choice if you want to prove your point.
philwelch 16y ago
The facts of the matter at that time were different from the facts of the matter when Scott Brown eventually won. When you come out and say it's only 20% probable for a given outcome to happen, and that outcome happens, it doesn't mean you were wrong, it just means a long shot happened. In fact, you would expect just about exactly 1 out of every 5 20% longshots to end up happening.