5 ms·

Yep. Beautifully wrought confirmations of bad hypotheses. After this election I'm taking anything stats-related on 538 (most everything) with a huge grain of

by 09bjb 10y ago

Yep. Beautifully wrought confirmations of bad hypotheses. After this election I'm taking anything stats-related on 538 (most everything) with a huge grain of salt. Can't stand another "Here's why the numbers show Hillary will win in a landslide" etc.

ivraatiems 10y ago
You do realize that of all the models available, 538's was the closest to the actual outcome and gave Trump by far the largest chance? 538 was very clear from the outset that the model's goal was predicting likelihoods, not being an oracle. It said there was a 1 in 3 chance of the election coming out like it did. That's not inaccurate - it was unlikely, but it still happened. Plus, 538 didn't make the data. It aggregated data others made. What exactly was bad in that hypothesis?
- jackhack 10y ago
  Compsci 101. Garbage in, garbage out.
  - ivraatiems 10y ago
    The polls weren't garbage. They were closer this year than they were in 2012. They were just slightly wrong in the wrong direction.
    - jackhack 10y ago
      It's not a pejorative. It's a common phrase meaning "the quality of your output is only as good as your input data". In this case, input data was skewed/biased/inaccurate for a number of reasons (sample choices, underreporting, etc.). Thus even the best predictions were wrong.
      - ivraatiems 10y ago
        That's true, but the trouble is that if you look back to past elections, those polls are also wrong in very similar ways. They were just wrong in a way that overemphasized the success of the right candidate, whereas polls this year were wrong in a way that suggested success for the wrong candidate. So the errors have existed for a long time, and in fact they were lesser this year than in previous years. Polls are getting better, not worse - but the narrative has become "never trust polls again because they were wrong" when the truth is closer to "learn lessons from the polls because they got close but made serious missteps."
      - nl 10y ago
        It's not a pejorative. It really kind of is actually. It's an implication that the output was garbage, and it clearly wasn't.
  - kirrent 10y ago
    That's only an issue if you don't account for uncertainty in your polling data. If you ever read their articles or listened to the model talk podcast you'd know that a huge amount of effort was put into trying to quantify the uncertainty in the polls based on historical polling errors and factors unique to this year.
labster 10y ago
I'm pretty sure 538 were the only ones not predicting a strong EC victory for Hillary. And they kept mentioning how terrible the polls were in Wisconsin and Michigan, and that they had to interpolate based on national polls. And in the popular vote Hillary won by a landslide. So this is a statistics package by the people who best interpreted the available evidence. Missing a 66% confidence forecast happens 1 out of every 3 times, it's not a big miss.
- deleted 10y ago
  [deleted]
nl 10y ago
Given that was the opposite of what 538 said I'm not sure you are think of the same site? Here is their final forecast: https://fivethirtyeight.com/features/final-election-update-theres-a-wide-range-of-outcomes-and-most-of-them-come-up-clinton/ https://fivethirtyeight.com/features/final-election-update-t... They gave Trump a 28.6% chance and pointed out that the model had an unusual amount of uncertainty. Given the lead Clinton ended up with in the popular vote and the small margin Trump won by in the critical states the prediction seems pretty reasonable to me. What do you think his chances should have been?
- gfosco 10y ago
  Closer to 50/50. It was obvious to me a year ago he was going to win. Go watch some pre-election videos of literally any news channel talking about the "path to 270". There are some incredible delusions at play. They could not separate their own feelings and desires from the work.
  - Houshalter 10y ago
    This is nonsense. They ran a statistical model - their feelings have nothing to do with it. They never tampered with the model to favor a particular candidate. They barely change the model at all, just feed it with polling data. Good job predicting the outcome of the election. But no way will you be able to do better than 538 in the long run.
  - nl 10y ago
    Go watch some pre-election videos of literally any news channel talking about the "path to 270". There are some incredible delusions at play. They could not separate their own feelings and desires from the work. I agree with this entirely (assuming you are talking generally, not about 538 specifically). There were people in the GJP Superforecasting project giving Clinton 98% chance of the win 2 months out, which in a 2-person race is just ridiculous (There were people giving Trump that as well, but a lot less of them). How did you come to this 50/50 prediction though? I can't find the report now, but I believe that Trump data science team gave him a 35% chance of winning, and that was only in the last 10 days of the campaign (they gave him much less prior to that). To go from the 28%/35% (538/Trump Team) range to 50% seems a pretty big jump. I haven't done the maths, but to make it a 50/50 doesn't that mean polls in Michigan etc would have had to be around 10% incorrect (to give him sufficient margin to make it that sure)? That seems a pretty big difference to what the error actually was.

by 09bjb 10y ago

ivraatiems 10y ago

You do realize that of all the models available, 538's was the closest to the actual outcome and gave Trump by far the largest chance? 538 was very clear from the outset that the model's goal was predicting likelihoods, not being an oracle. It said there was a 1 in 3 chance of the election coming out like it did. That's not inaccurate - it was unlikely, but it still happened. Plus, 538 didn't make the data. It aggregated data others made. What exactly was bad in that hypothesis?

jackhack 10y ago
Compsci 101. Garbage in, garbage out.
- ivraatiems 10y ago
  The polls weren't garbage. They were closer this year than they were in 2012. They were just slightly wrong in the wrong direction.
  - jackhack 10y ago
    It's not a pejorative. It's a common phrase meaning "the quality of your output is only as good as your input data". In this case, input data was skewed/biased/inaccurate for a number of reasons (sample choices, underreporting, etc.). Thus even the best predictions were wrong.
    - ivraatiems 10y ago
      That's true, but the trouble is that if you look back to past elections, those polls are also wrong in very similar ways. They were just wrong in a way that overemphasized the success of the right candidate, whereas polls this year were wrong in a way that suggested success for the wrong candidate. So the errors have existed for a long time, and in fact they were lesser this year than in previous years. Polls are getting better, not worse - but the narrative has become "never trust polls again because they were wrong" when the truth is closer to "learn lessons from the polls because they got close but made serious missteps."
    - nl 10y ago
      It's not a pejorative. It really kind of is actually. It's an implication that the output was garbage, and it clearly wasn't.
- kirrent 10y ago
  That's only an issue if you don't account for uncertainty in your polling data. If you ever read their articles or listened to the model talk podcast you'd know that a huge amount of effort was put into trying to quantify the uncertainty in the polls based on historical polling errors and factors unique to this year.

labster 10y ago

I'm pretty sure 538 were the only ones not predicting a strong EC victory for Hillary. And they kept mentioning how terrible the polls were in Wisconsin and Michigan, and that they had to interpolate based on national polls. And in the popular vote Hillary won by a landslide. So this is a statistics package by the people who best interpreted the available evidence. Missing a 66% confidence forecast happens 1 out of every 3 times, it's not a big miss.

deleted 10y ago
[deleted]

nl 10y ago

Given that was the opposite of what 538 said I'm not sure you are think of the same site? Here is their final forecast: https://fivethirtyeight.com/features/final-election-update-theres-a-wide-range-of-outcomes-and-most-of-them-come-up-clinton/ https://fivethirtyeight.com/features/final-election-update-t... They gave Trump a 28.6% chance and pointed out that the model had an unusual amount of uncertainty. Given the lead Clinton ended up with in the popular vote and the small margin Trump won by in the critical states the prediction seems pretty reasonable to me. What do you think his chances should have been?

gfosco 10y ago
Closer to 50/50. It was obvious to me a year ago he was going to win. Go watch some pre-election videos of literally any news channel talking about the "path to 270". There are some incredible delusions at play. They could not separate their own feelings and desires from the work.
- Houshalter 10y ago
  This is nonsense. They ran a statistical model - their feelings have nothing to do with it. They never tampered with the model to favor a particular candidate. They barely change the model at all, just feed it with polling data. Good job predicting the outcome of the election. But no way will you be able to do better than 538 in the long run.
- nl 10y ago
  Go watch some pre-election videos of literally any news channel talking about the "path to 270". There are some incredible delusions at play. They could not separate their own feelings and desires from the work. I agree with this entirely (assuming you are talking generally, not about 538 specifically). There were people in the GJP Superforecasting project giving Clinton 98% chance of the win 2 months out, which in a 2-person race is just ridiculous (There were people giving Trump that as well, but a lot less of them). How did you come to this 50/50 prediction though? I can't find the report now, but I believe that Trump data science team gave him a 35% chance of winning, and that was only in the last 10 days of the campaign (they gave him much less prior to that). To go from the 28%/35% (538/Trump Team) range to 50% seems a pretty big jump. I haven't done the maths, but to make it a 50/50 doesn't that mean polls in Michigan etc would have had to be around 10% incorrect (to give him sufficient margin to make it that sure)? That seems a pretty big difference to what the error actually was.