10 ms·

I think Taleb’s point is a bit more damning — implying the software does not do what it says it would. If all sorts of things can happen, where you predictions

I think Taleb’s point is a bit more damning — implying the software does not do what it says it would. If all sorts of things can happen, where you predictions change widely, Taleb’s argument is that you can’t use a number like 90%. You have to include the error, which ends up saying “50%” for both before the election. This doesn’t mean that all prediction suffers from this effect, and you have to be skeptical everywhere. I don’t think that’s a fair characterization. Taleb has won by mispriced options. His main statement is that the true probability of this particular event is much closer to 50%

cyphar 6y ago
That's what confidence intervals are for. You cannot condense both your expectation value and uncertainty in a single percentage (it is not correct to say that the probability of an outcome is 50% if 80% of likely outcomes result in one candidate winning). Otherwise you'll end up with every 5-week weather forecast saying there's a 50% chance of rain.
- stopachka 6y ago
  Indeed. Taleb’s point if I understand it correctly is that the uncertainty in this so high that the number comes realistically much closer to 50%.
  - cyphar 6y ago
    We're not in agreement here -- the solution to uncertainty in statistics is to model the uncertainty and treat them as confidence intervals, not throw your hands up and give everything a coin-flip probability. The probability isn't "realistically much closer to 50%" -- that's not how uncertainties in statistics work. It is possible that (if the uncertainty in the model was smaller) that the "real" probability is 50% but there's no way of knowing if that is the case.
    - stopachka 6y ago
      We can try going deeper. I may be making a dumb mistake: when I got to 538 — the main number I see is 89%. I don’t see 89% +- 40. What is his confidence interval?
      - cyphar 6y ago
        If you want a more data-heavy version go to the graphs, then click on the "electoral votes" tab. The shaded part is the 80% confidence interval of each prediction. This is the historical graph form of the "every outcome in our simulations" chart underneath the dot chart. It should be noted that the dot chart at the top of the page does kind of outline the confidence interval (giving a realistic example of what kinds of outcomes might happen) but is obviously geared towards lay-people who don't know what a confidence interval is. > We simulate the election 40,000 times to see who wins most often. The sample of 100 outcomes below gives you a good idea of the range of scenarios our model thinks is possible. I think it would be better if the confidence intervals were better signposted, but it's possible that only the electoral college win margins have solid confidence intervals (I'll admit I don't know enough statistics to know if you could trivially transform the confidence intervals of the electoral college votes to win percentage). The senate and house predictions explicitly show the confidence interval in their main graphs.
        stopachka 6y ago
        I think if you see 90% with 80% confidence interval, you can intuit that he isn't counting uncertainty in the tails. Consider, Would you be comfortable making a bet with someone, on a 90/10 payoff? This, I think is Taleb's central point. Silver's algorithm is not counting the uncertainty correctly.
      - dragonwriter 6y ago
        The 89% probability of victory is the result of the actual predicted electoral vote total and the uncertainty in that prediction. Placing an additional uncertainty on the uncertainty is...very much not understanding uncertainty.
  - awb 6y ago
    Well, he's right in the sense that there's close to a 50% chance of a swing in either direction (blunders, dirt, health issues, polling errors, etc). However, when one candidate is up 8+% (even months out) against an incumbent with 4 years of net negative approval ratings, to call that a toss up is like saying the frontrunner is substantially more likely to blunder than the underdog. As Silver points out however, a Trump win is just as likely as the largest Democratic landslide victory in modern history. The polls could move either way, but the only way they move enough for Trump to win is a larger than 2016 polling error in his favor in all 7 battleground states, which doesn't sound like a 50% chance at all.
    - cameldrv 6y ago
      To say Biden is up by 8 points is missing a lot of microstructure. In every state Trump needs except PA, he’s within about 2 points. In PA, the critical state, he’s down by about 4.5 points, but Biden’s lead has been diminishing rapidly in the past two weeks. These polls also rely on a turnout model. There’s no way to calibrate these models for a pandemic election where some people may be more scared to vote than others. I give Biden the edge but it’s nowhere near 90/10.
      - rurban 6y ago
        I don't know where you got your numbers from. I got them from Tanenbaums site https://www.electoral-vote.com/evp2020/Pres/Graphs/all.html https://www.electoral-vote.com/evp2020/Pres/Graphs/all.html and this shows that Trump would win all swing states and thus the election, if there was not the surprising swing in Texas from Rep to Dem lately, which broke Trump. You cannot really trust Tanenbaums summary as heavy Dem fanboy, which was wrong last time and is wrong this time, but the raw polling numbers are showing a clear picture. Early votes missed the Hunter Biden scandal, and Texas was champion in early votes.
- dragonwriter 6y ago
  > You cannot condense both your expectation value and uncertainty in a single percentage Yes, you can. The predicted outcome is a particular electoral vote total. The uncertainty around that predicted value is what gets you from that to a probability that the outcome will be at least 270. That's a single percentage that results from the combination of the predicted result and the uncertainty around it.
dragonwriter 6y ago
> If all sorts of things can happen, where you predictions change widely, Taleb’s argument is that you can’t use a number like 90%. You have to include the error, which ends up saying “50%” for both before the election. Taleb is, simply, wrong. The difference from 100%/0% is the uncertainty. If new information confirms rather than contradicts prior information, that uncertainty goes down, if it contradicts it, the uncertainty goes up. You expect most of the time the general trend over time to be declining uncertainty, but when there is a period of continuing new information at odds with the prior information, you get a period of increasing uncertainty. So if you had a very large number of Presidential election cycles with the same model, you'd expect most of them to generally trend toward greater certainty over time, but you'd expect a few of them to have extended periods of declining certainty. Does Silver's Presidential model work that way? It's hard to tell. One, because there aren't a lot of cycles to look at, and because they aren't the exact same model, is it's not quite what you'd want to look at it to assess that. Well, 2012 and 2020 have had very much the general shape of growing certainty you'd expect, while 2016 didn't.
- coldtea 6y ago
  >Taleb is, simply, wrong. Just not in any mathematics...

cyphar 6y ago

That's what confidence intervals are for. You cannot condense both your expectation value and uncertainty in a single percentage (it is not correct to say that the probability of an outcome is 50% if 80% of likely outcomes result in one candidate winning). Otherwise you'll end up with every 5-week weather forecast saying there's a 50% chance of rain.

stopachka 6y ago
Indeed. Taleb’s point if I understand it correctly is that the uncertainty in this so high that the number comes realistically much closer to 50%.
- cyphar 6y ago
  We're not in agreement here -- the solution to uncertainty in statistics is to model the uncertainty and treat them as confidence intervals, not throw your hands up and give everything a coin-flip probability. The probability isn't "realistically much closer to 50%" -- that's not how uncertainties in statistics work. It is possible that (if the uncertainty in the model was smaller) that the "real" probability is 50% but there's no way of knowing if that is the case.
  - stopachka 6y ago
    We can try going deeper. I may be making a dumb mistake: when I got to 538 — the main number I see is 89%. I don’t see 89% +- 40. What is his confidence interval?
    - cyphar 6y ago
      If you want a more data-heavy version go to the graphs, then click on the "electoral votes" tab. The shaded part is the 80% confidence interval of each prediction. This is the historical graph form of the "every outcome in our simulations" chart underneath the dot chart. It should be noted that the dot chart at the top of the page does kind of outline the confidence interval (giving a realistic example of what kinds of outcomes might happen) but is obviously geared towards lay-people who don't know what a confidence interval is. > We simulate the election 40,000 times to see who wins most often. The sample of 100 outcomes below gives you a good idea of the range of scenarios our model thinks is possible. I think it would be better if the confidence intervals were better signposted, but it's possible that only the electoral college win margins have solid confidence intervals (I'll admit I don't know enough statistics to know if you could trivially transform the confidence intervals of the electoral college votes to win percentage). The senate and house predictions explicitly show the confidence interval in their main graphs.
      - stopachka 6y ago
        I think if you see 90% with 80% confidence interval, you can intuit that he isn't counting uncertainty in the tails. Consider, Would you be comfortable making a bet with someone, on a 90/10 payoff? This, I think is Taleb's central point. Silver's algorithm is not counting the uncertainty correctly.
    - dragonwriter 6y ago
      The 89% probability of victory is the result of the actual predicted electoral vote total and the uncertainty in that prediction. Placing an additional uncertainty on the uncertainty is...very much not understanding uncertainty.
- awb 6y ago
  Well, he's right in the sense that there's close to a 50% chance of a swing in either direction (blunders, dirt, health issues, polling errors, etc). However, when one candidate is up 8+% (even months out) against an incumbent with 4 years of net negative approval ratings, to call that a toss up is like saying the frontrunner is substantially more likely to blunder than the underdog. As Silver points out however, a Trump win is just as likely as the largest Democratic landslide victory in modern history. The polls could move either way, but the only way they move enough for Trump to win is a larger than 2016 polling error in his favor in all 7 battleground states, which doesn't sound like a 50% chance at all.
  - cameldrv 6y ago
    To say Biden is up by 8 points is missing a lot of microstructure. In every state Trump needs except PA, he’s within about 2 points. In PA, the critical state, he’s down by about 4.5 points, but Biden’s lead has been diminishing rapidly in the past two weeks. These polls also rely on a turnout model. There’s no way to calibrate these models for a pandemic election where some people may be more scared to vote than others. I give Biden the edge but it’s nowhere near 90/10.
    - rurban 6y ago
      I don't know where you got your numbers from. I got them from Tanenbaums site https://www.electoral-vote.com/evp2020/Pres/Graphs/all.html https://www.electoral-vote.com/evp2020/Pres/Graphs/all.html and this shows that Trump would win all swing states and thus the election, if there was not the surprising swing in Texas from Rep to Dem lately, which broke Trump. You cannot really trust Tanenbaums summary as heavy Dem fanboy, which was wrong last time and is wrong this time, but the raw polling numbers are showing a clear picture. Early votes missed the Hunter Biden scandal, and Texas was champion in early votes.
dragonwriter 6y ago
> You cannot condense both your expectation value and uncertainty in a single percentage Yes, you can. The predicted outcome is a particular electoral vote total. The uncertainty around that predicted value is what gets you from that to a probability that the outcome will be at least 270. That's a single percentage that results from the combination of the predicted result and the uncertainty around it.

dragonwriter 6y ago

> If all sorts of things can happen, where you predictions change widely, Taleb’s argument is that you can’t use a number like 90%. You have to include the error, which ends up saying “50%” for both before the election. Taleb is, simply, wrong. The difference from 100%/0% is the uncertainty. If new information confirms rather than contradicts prior information, that uncertainty goes down, if it contradicts it, the uncertainty goes up. You expect most of the time the general trend over time to be declining uncertainty, but when there is a period of continuing new information at odds with the prior information, you get a period of increasing uncertainty. So if you had a very large number of Presidential election cycles with the same model, you'd expect most of them to generally trend toward greater certainty over time, but you'd expect a few of them to have extended periods of declining certainty. Does Silver's Presidential model work that way? It's hard to tell. One, because there aren't a lot of cycles to look at, and because they aren't the exact same model, is it's not quite what you'd want to look at it to assess that. Well, 2012 and 2020 have had very much the general shape of growing certainty you'd expect, while 2016 didn't.

coldtea 6y ago
>Taleb is, simply, wrong. Just not in any mathematics...