8 ms·

Nate Silver got lucky one time and has been coasting on it since. The man has no credibility.

by mdoms 6y ago

garmaine 6y ago
He’s been consistently more right (less wrong?) than the mainstream media. This hasn’t been a one off thing.
- dragonwriter 6y ago
  > He’s been consistently more right (less wrong?) than the mainstream media. Well, once he rose to popularity, 538 has been part of the mainstream media, affiliated with the NYTimes, then owned by ESPN, then transferred within the Disney Empire to ABC News. But, sure, it's pretty consistently been one of the better (not just election) statistical forecasting shops in the mainstream media.
- hoorayimhelping 6y ago
  That's the whole point, I think. We're saying, "he's better at predicting things than the mainstream media!" The issue is that the mainstream media's bar is so low, there is no bar. And the reason this guy is somewhat respected is because his bar is only slightly higher than rat shit, which seems amazing compared to what we're used to. The guy sucks at actually modelling outcomes. Because everyone does. He just sucks very slightly less than everyone else. Maybe the issue isn't that Nate Silver is slightly better than average, maybe the issue is that we shouldn't be sharing this garbage on the news, since it's not reflective of reality. It's like saying, "sure, Cuba is a human rights nightmare, but it's okay to trade and interact with them, cause at least it ain't the Congo!"
  - asdfasgasdgasdg 6y ago
    Asking for people to not make or publish predictions is asking for the sun to not rise. You'll never see that day. Being less wrong is the best option that is actually on the table.
fabian2k 6y ago
Some of the other models for the last election were really bad. Giving something like a 95-98% chance to Hillary was arguably a fundamental failure. I found it very odd how they arrived at those numbers by e.g. treating every state as a separate chance while mostly ignoring that those results are not uncorrelated. I think he does a better job at emphasizing the uncertainty while still showing that polls can be pretty reliable.
- sixothree 6y ago
  It also discounts the penetration of board of elections in some 30+ states.
- thinkloop 6y ago
  > Giving something like a 95-98% chance to Hillary was arguably a fundamental failure Why was it a fundamental failure? A 5% chance is one in twenty, it happens.
  - fabian2k 6y ago
    Im not saying that because of the percentage alone, but because I think the methodology there was suspect. And this was criticized before the election. The lead in the polls was not large, and making it look like a certain thing by treating the states as almost independent results just doesn't make any sense to me.
    - deleted 6y ago
      [deleted]
  - richardw 6y ago
    You think if Hillary went up against Trump, America would choose her 19 times out of 20? I’m not sure that makes the mental calculation any better than 95%. 95% is “Obama vs a dog”. Maybe in another country. Every 20 elections, Obama is bitten by the dog and doesn’t make it.
    - 6y ago
Dumblydorr 6y ago
I don't agree, I think he speaks and writes with nuance and intelligence. What is your issue with his Bayesian statistical approach? He went to UofC and LSE so he clearly had top notch training. His 2016 model and he himself were much more predictive of the trump EC win, he repeatedly stated it was a possible outcome, something the vast majority of other forecasters completely missed.
- ceilingcorner 6y ago
  Nate Silver was merely wrong, as compared to everyone else, who were spectacularly wrong. I'm not sure how that counts as a ringing endorsement. These polls also never factor in things like "social acceptability of admitting that one voted for an unpopular candidate" or "groupthink among media organizations which aligns to their side of the isle." The entire polling fiasco should be interpreted as the limitations of quantitative data, as opposed to qualitative data. Our final forecast, issued early Tuesday evening, had Trump with a 29 percent chance of winning the Electoral College.1 By comparison, other models tracked by The New York Times put Trump’s odds at: 15 percent, 8 percent, 2 percent and less than 1 percent. And betting markets put Trump’s chances at just 18 percent at midnight on Tuesday, when Dixville Notch, New Hampshire, cast its votes. https://fivethirtyeight.com/features/why-fivethirtyeight-gave-trump-a-better-chance-than-almost-anyone-else/ https://fivethirtyeight.com/features/why-fivethirtyeight-gav...
  - sdenton4 6y ago
    "never factor in things..." Good luck measuring those. Consider this on the individual poller level. Every four years you get a new batch of pundit driven ideas about what "really" driving the voters. Suppose you add a question that's meant to magically reveal hidden voter preferences that are hidden by shame. What do the results of that question mean for the bottom line? Well, you're probably going to need multiple election cycle to find out. And by the time you do, the pundit have a new pile of bullshit for you to implement. Now take it up a level. You've got a bunch of pollsters of caring predictive quality. Let them figure out what works best and include an estimate of their quality in how you process their results. I have no idea is pollster X's new question about cheese preferences is predictive, and honestly neither do they until a couple more cycles pass. All you can do is wight by past performance. KISS wins in these situations.
bitxbit 6y ago
Nate likes to make pretty charts. Let’s just leave it at that. Anyone who’s even half serious about econometrics or statistics can see right through his work.
- evgen 6y ago
  Sure they can, and they can do so much better that their predictions are trusted to the point where they become a part of the narrative. Oh yeah, guess not...
- person_of_color 6y ago
  What are some serious forecasting resources?
- djohnston 6y ago
  where are the better models?
- take_a_breath 6y ago
  Where is your work published? I’d love to look through your predictions.
- scrollaway 6y ago
  > Let’s just leave it at that. Let's not; let's instead make a reasoned argument explaining why you think that. As is, your comment is just flamebait.
IfOnlyYouKnew 6y ago
I believe they got every single state right in three consecutive elections before 2016? And even in 2016, their predictions align reasonably well with the outcome, easily beating all competitors. ..and that's only presidential elections. See https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/ for their quantitative reflections on accuracy. Looks pretty good.
fullshark 6y ago
I was very unimpressed with his democratic primary model, which he basically fine tuned every week when new data came in in order to get a result that was closer to his own preconceptions of the race.
- glial 6y ago
  > fine tuned every week when new data came in Yeah, that’s what you’re supposed to do - use new evidence to update your beliefs.
  - fullshark 6y ago
    I don't mean he allowed the model to update from new results, I mean the model would update, he'd be unhappy by its results so he'd change it so the outcome reflected his own biases. If you do that you produce a bunch of mathematical rationalizations for probability distributions that you invented from scratch.
    - glial 6y ago
      Ach, thanks for clarifying.
tunesmith 6y ago
For anyone with a history of probabilistic predictions, the goal is to be well-calibrated. For predictions that have 70% likelihood, you want to be right 70% of the time. 90% likelihood, you want to be right 90% of the time. There's a mathematical way to analyze an entire set of predictions and determine how well-calibrated that set of predictions is. Silver claims 538's predictions are well-calibrated. There are also tools and websites out there that people can use to make predictions and track their own calibration over time. They're pretty fun in terms of encouraging one's own sense of rationality.
iainmerrick 6y ago
Which “one time” were you thinking of?

Nate Silver got lucky one time and has been coasting on it since. The man has no credibility.

by mdoms 6y ago

Nate Silver got lucky one time and has been coasting on it since. The man has no credibility.

garmaine 6y ago

He’s been consistently more right (less wrong?) than the mainstream media. This hasn’t been a one off thing.

dragonwriter 6y ago
> He’s been consistently more right (less wrong?) than the mainstream media. Well, once he rose to popularity, 538 has been part of the mainstream media, affiliated with the NYTimes, then owned by ESPN, then transferred within the Disney Empire to ABC News. But, sure, it's pretty consistently been one of the better (not just election) statistical forecasting shops in the mainstream media.
hoorayimhelping 6y ago
That's the whole point, I think. We're saying, "he's better at predicting things than the mainstream media!" The issue is that the mainstream media's bar is so low, there is no bar. And the reason this guy is somewhat respected is because his bar is only slightly higher than rat shit, which seems amazing compared to what we're used to. The guy sucks at actually modelling outcomes. Because everyone does. He just sucks very slightly less than everyone else. Maybe the issue isn't that Nate Silver is slightly better than average, maybe the issue is that we shouldn't be sharing this garbage on the news, since it's not reflective of reality. It's like saying, "sure, Cuba is a human rights nightmare, but it's okay to trade and interact with them, cause at least it ain't the Congo!"
- asdfasgasdgasdg 6y ago
  Asking for people to not make or publish predictions is asking for the sun to not rise. You'll never see that day. Being less wrong is the best option that is actually on the table.

fabian2k 6y ago

Some of the other models for the last election were really bad. Giving something like a 95-98% chance to Hillary was arguably a fundamental failure. I found it very odd how they arrived at those numbers by e.g. treating every state as a separate chance while mostly ignoring that those results are not uncorrelated. I think he does a better job at emphasizing the uncertainty while still showing that polls can be pretty reliable.

sixothree 6y ago
It also discounts the penetration of board of elections in some 30+ states.
thinkloop 6y ago
> Giving something like a 95-98% chance to Hillary was arguably a fundamental failure Why was it a fundamental failure? A 5% chance is one in twenty, it happens.
- fabian2k 6y ago
  Im not saying that because of the percentage alone, but because I think the methodology there was suspect. And this was criticized before the election. The lead in the polls was not large, and making it look like a certain thing by treating the states as almost independent results just doesn't make any sense to me.
  - deleted 6y ago
    [deleted]
- richardw 6y ago
  You think if Hillary went up against Trump, America would choose her 19 times out of 20? I’m not sure that makes the mental calculation any better than 95%. 95% is “Obama vs a dog”. Maybe in another country. Every 20 elections, Obama is bitten by the dog and doesn’t make it.
  - 6y ago

Dumblydorr 6y ago

I don't agree, I think he speaks and writes with nuance and intelligence. What is your issue with his Bayesian statistical approach? He went to UofC and LSE so he clearly had top notch training. His 2016 model and he himself were much more predictive of the trump EC win, he repeatedly stated it was a possible outcome, something the vast majority of other forecasters completely missed.

ceilingcorner 6y ago
Nate Silver was merely wrong, as compared to everyone else, who were spectacularly wrong. I'm not sure how that counts as a ringing endorsement. These polls also never factor in things like "social acceptability of admitting that one voted for an unpopular candidate" or "groupthink among media organizations which aligns to their side of the isle." The entire polling fiasco should be interpreted as the limitations of quantitative data, as opposed to qualitative data. Our final forecast, issued early Tuesday evening, had Trump with a 29 percent chance of winning the Electoral College.1 By comparison, other models tracked by The New York Times put Trump’s odds at: 15 percent, 8 percent, 2 percent and less than 1 percent. And betting markets put Trump’s chances at just 18 percent at midnight on Tuesday, when Dixville Notch, New Hampshire, cast its votes. https://fivethirtyeight.com/features/why-fivethirtyeight-gave-trump-a-better-chance-than-almost-anyone-else/ https://fivethirtyeight.com/features/why-fivethirtyeight-gav...
- sdenton4 6y ago
  "never factor in things..." Good luck measuring those. Consider this on the individual poller level. Every four years you get a new batch of pundit driven ideas about what "really" driving the voters. Suppose you add a question that's meant to magically reveal hidden voter preferences that are hidden by shame. What do the results of that question mean for the bottom line? Well, you're probably going to need multiple election cycle to find out. And by the time you do, the pundit have a new pile of bullshit for you to implement. Now take it up a level. You've got a bunch of pollsters of caring predictive quality. Let them figure out what works best and include an estimate of their quality in how you process their results. I have no idea is pollster X's new question about cheese preferences is predictive, and honestly neither do they until a couple more cycles pass. All you can do is wight by past performance. KISS wins in these situations.

bitxbit 6y ago

Nate likes to make pretty charts. Let’s just leave it at that. Anyone who’s even half serious about econometrics or statistics can see right through his work.

evgen 6y ago
Sure they can, and they can do so much better that their predictions are trusted to the point where they become a part of the narrative. Oh yeah, guess not...
person_of_color 6y ago
What are some serious forecasting resources?
djohnston 6y ago
where are the better models?
take_a_breath 6y ago
Where is your work published? I’d love to look through your predictions.
scrollaway 6y ago
> Let’s just leave it at that. Let's not; let's instead make a reasoned argument explaining why you think that. As is, your comment is just flamebait.

IfOnlyYouKnew 6y ago

I believe they got every single state right in three consecutive elections before 2016? And even in 2016, their predictions align reasonably well with the outcome, easily beating all competitors. ..and that's only presidential elections. See https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/ for their quantitative reflections on accuracy. Looks pretty good.

fullshark 6y ago

I was very unimpressed with his democratic primary model, which he basically fine tuned every week when new data came in in order to get a result that was closer to his own preconceptions of the race.

glial 6y ago
> fine tuned every week when new data came in Yeah, that’s what you’re supposed to do - use new evidence to update your beliefs.
- fullshark 6y ago
  I don't mean he allowed the model to update from new results, I mean the model would update, he'd be unhappy by its results so he'd change it so the outcome reflected his own biases. If you do that you produce a bunch of mathematical rationalizations for probability distributions that you invented from scratch.
  - glial 6y ago
    Ach, thanks for clarifying.

tunesmith 6y ago

For anyone with a history of probabilistic predictions, the goal is to be well-calibrated. For predictions that have 70% likelihood, you want to be right 70% of the time. 90% likelihood, you want to be right 90% of the time. There's a mathematical way to analyze an entire set of predictions and determine how well-calibrated that set of predictions is. Silver claims 538's predictions are well-calibrated. There are also tools and websites out there that people can use to make predictions and track their own calibration over time. They're pretty fun in terms of encouraging one's own sense of rationality.

iainmerrick 6y ago

Which “one time” were you thinking of?