5 ms·

Loads of people criticized Silver for not having "predicted" Trump's electoral victory. I imagine Silver's language was trying to hand-hold people to an unders

by tcbawo 8y ago

Loads of people criticized Silver for not having "predicted" Trump's electoral victory. I imagine Silver's language was trying to hand-hold people to an understanding that even 20% probability events happen 1 out of 5 times. With so many people trying to twist and spin every headline, prediction, number, I appreciate Silver's attempt to make a reasonable guess with transparent methodologies.

jandrese 8y ago
The same people were criticizing him a day before because everybody else had Hillary at a 99% chance to win. Nate tweets that Trump is one standard polling error away from winning and people scoffed.
Someone1234 8y ago
It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but people infer from the final datapoint to the conclusion later that same day about 538's accuracy. What I am saying is: 538 needs to keep a historical record of their predictions.
- wtallis 8y ago
  538 published this last week: https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ https://fivethirtyeight.com/features/when-we-say-70-percent-...
- dandelany 8y ago
  Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It’s still visible here: https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/#change https://projects.fivethirtyeight.com/2018-midterm-election-f...
  - dragonwriter 8y ago
    > Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It has not changed. They did this the whole way back when it was an independent blog before the NY Times (later ESPN then ABC News) picked them up.
    - Someone1234 8y ago
      Where are they? Five different posters have claimed this, but zero have been able to link to the historical predictions.
      - dragonwriter 8y ago
        > Where are they? Five different posters have claimed this, but zero have been able to link to the historical predictions. They are literally charted on the page with the current prediction, e.g., for the 2016 Presidential race (which is locked on the last forecast): https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/ , under “How the forecast has changed”, it charts the forecast (by default, the win probability) for the entire time the race was being forecast.
- KC8ZKF 8y ago
  The 2016 election predictions are still posted. https://projects.fivethirtyeight.com/2016-election-forecast/?ex_cid=2016-forecast https://projects.fivethirtyeight.com/2016-election-forecast/...
  - Someone1234 8y ago
    Yes the newest one. Not the historical ones.
    - hencq 8y ago
      Maybe I misunderstand your argument, but doesn't that linked page have a section "How the forecast has changed"[1]? [1] https://projects.fivethirtyeight.com/2016-election-forecast/?ex_cid=2016-forecast#odds https://projects.fivethirtyeight.com/2016-election-forecast/...
- dragonwriter 8y ago
  > It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. It's worth noting that you are completely wrong: not only do they keep such a record, they also provide a handy chart of it on the latest prediction page so you can easily see exactly how the prediction changed over time.
  - Someone1234 8y ago
    Where? Go ahead and link to it.
    - tweezy 8y ago
      Hey I don't have a dog in this fight, but I think this is the link you're looking for. https://github.com/fivethirtyeight/checking-our-work-data https://github.com/fivethirtyeight/checking-our-work-data
      - Someone1234 8y ago
        Nope. I'm looking for 538's predictions. You cannot go from the raw data they used back to how they interpreted it.
        tweezy 8y ago
        Please look at the actual data on the linked page. Each of the csv files on that page has thousands of predictions 538 made going back as far as 2008. This isn't the raw data they used to make the predictions, these are the actual predications that they made. Edit: For example, here are the headers on the file named presidential_elections.csv year,office,state,district,election_date,forecast_date,forecast_type,party,candidate,projected_voteshare,actual_voteshare,probwin,probwin_outcome These are the actual forecasts they made for presidential elections going all the way back to 2008.
        wilg 8y ago
bambax 8y ago
I have a hard time understanding this: what does it mean to assign probabilities to one-off events?? The only way to verify probabilities predictions is to test it a great number of times. Otherwise you can say anything and never be wrong (except for 0 or 100%).
- deleted 8y ago
  [deleted]
- roenxi 8y ago
  Those questions underpin big chunks of philosophy. Assigning a probability to a one-off event is enumerating all the ways it could happen, all the ways it could not happen and assigning a probability to each of those ways. Obviously there is a lot of guesswork; but if you need to make a decision based on the future that approach gives you a much better chance of making a good decision. In practice an event will be made up of components that are more predictable than the whole, and some real uncertainties. There is a lot to be gained by thinking hard about the situation, and assigning a probability will do that. Silly example - how do I estimate my risk of falling climbing up a set of stairs that I've never climbed before? * Baseline risk of tripping - I have a lifetime of data. * Increase of risk being on a staircase - I have an area I want to put my foot on (a specific step) that is about 1/3 of the area that my foot usually falls in, so that increases the risk by an amount that can be reasonably estimated. * I will watch my foot - maybe an order of magnitude improvement in precision. This is enough to let me estimate the risk of carrying a bulky object (that obscures my view of my feet) up a staircase. I've isolated the uncertainty (how much does visual observation change the odds) from the certainties (areas, background rate). Now I can take that to several experts who will identify new mechanisms and tighten up my estimations on how big a deal the components are. In this way - even though the final % I come to would still be a bit arbitrary - it is starting to become a summary of what a large number of people think about the inputs to the problem and their relative magnitudes. Being able to communicate all that thinking with a single number is a miracle in its own way.
- mr_toad 8y ago
  It’s not that the election itself is subject to random uncertainty so much as the data that they are using to make the prediction is. The data might favour one candidate, but even assuming it is unbiased and representative, it is only a random sample, and there is a chance it could be randomly wrong.
seambot1 8y ago
I remember him saying before the election that the odds of Trump winning we're roughly the odds of a randomly picked day being a Saturday. Which always seemed pretty clear to me - Saturdays definitely happen, Trump could definitely happen. And then it did, and it was nothing like a Saturday, not at all.

by tcbawo 8y ago

jandrese 8y ago

The same people were criticizing him a day before because everybody else had Hillary at a 99% chance to win. Nate tweets that Trump is one standard polling error away from winning and people scoffed.

Someone1234 8y ago

It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. 538 also predicted lower chances of victory in the weeks and months leading up to that, then adjusted it on election day vanishing away their earlier guesses. This isn't wrong on the face of it, but people infer from the final datapoint to the conclusion later that same day about 538's accuracy. What I am saying is: 538 needs to keep a historical record of their predictions.

wtallis 8y ago
538 published this last week: https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ https://fivethirtyeight.com/features/when-we-say-70-percent-...
dandelany 8y ago
Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It’s still visible here: https://projects.fivethirtyeight.com/2018-midterm-election-forecast/house/#change https://projects.fivethirtyeight.com/2018-midterm-election-f...
- dragonwriter 8y ago
  > Maybe this was true of past elections, but their 2018 house forecast had a “how this forecast has changed” section that showed their historical predictions throughout election season. It has not changed. They did this the whole way back when it was an independent blog before the NY Times (later ESPN then ABC News) picked them up.
  - Someone1234 8y ago
    Where are they? Five different posters have claimed this, but zero have been able to link to the historical predictions.
    - dragonwriter 8y ago
      > Where are they? Five different posters have claimed this, but zero have been able to link to the historical predictions. They are literally charted on the page with the current prediction, e.g., for the 2016 Presidential race (which is locked on the last forecast): https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/ , under “How the forecast has changed”, it charts the forecast (by default, the win probability) for the entire time the race was being forecast.
KC8ZKF 8y ago
The 2016 election predictions are still posted. https://projects.fivethirtyeight.com/2016-election-forecast/?ex_cid=2016-forecast https://projects.fivethirtyeight.com/2016-election-forecast/...
- Someone1234 8y ago
  Yes the newest one. Not the historical ones.
  - hencq 8y ago
    Maybe I misunderstand your argument, but doesn't that linked page have a section "How the forecast has changed"[1]? [1] https://projects.fivethirtyeight.com/2016-election-forecast/?ex_cid=2016-forecast#odds https://projects.fivethirtyeight.com/2016-election-forecast/...
dragonwriter 8y ago
> It is worth noting that 538 doesn't keep a historical record of their predictions, making it difficult to really see how they predict outcomes coming up to a conclusion. You can only see the latest/final prediction. It's worth noting that you are completely wrong: not only do they keep such a record, they also provide a handy chart of it on the latest prediction page so you can easily see exactly how the prediction changed over time.
- Someone1234 8y ago
  Where? Go ahead and link to it.
  - tweezy 8y ago
    Hey I don't have a dog in this fight, but I think this is the link you're looking for. https://github.com/fivethirtyeight/checking-our-work-data https://github.com/fivethirtyeight/checking-our-work-data
    - Someone1234 8y ago
      Nope. I'm looking for 538's predictions. You cannot go from the raw data they used back to how they interpreted it.
      - tweezy 8y ago
        Please look at the actual data on the linked page. Each of the csv files on that page has thousands of predictions 538 made going back as far as 2008. This isn't the raw data they used to make the predictions, these are the actual predications that they made. Edit: For example, here are the headers on the file named presidential_elections.csv year,office,state,district,election_date,forecast_date,forecast_type,party,candidate,projected_voteshare,actual_voteshare,probwin,probwin_outcome These are the actual forecasts they made for presidential elections going all the way back to 2008.
      - wilg 8y ago

bambax 8y ago

I have a hard time understanding this: what does it mean to assign probabilities to one-off events?? The only way to verify probabilities predictions is to test it a great number of times. Otherwise you can say anything and never be wrong (except for 0 or 100%).

deleted 8y ago
[deleted]
roenxi 8y ago
Those questions underpin big chunks of philosophy. Assigning a probability to a one-off event is enumerating all the ways it could happen, all the ways it could not happen and assigning a probability to each of those ways. Obviously there is a lot of guesswork; but if you need to make a decision based on the future that approach gives you a much better chance of making a good decision. In practice an event will be made up of components that are more predictable than the whole, and some real uncertainties. There is a lot to be gained by thinking hard about the situation, and assigning a probability will do that. Silly example - how do I estimate my risk of falling climbing up a set of stairs that I've never climbed before? * Baseline risk of tripping - I have a lifetime of data. * Increase of risk being on a staircase - I have an area I want to put my foot on (a specific step) that is about 1/3 of the area that my foot usually falls in, so that increases the risk by an amount that can be reasonably estimated. * I will watch my foot - maybe an order of magnitude improvement in precision. This is enough to let me estimate the risk of carrying a bulky object (that obscures my view of my feet) up a staircase. I've isolated the uncertainty (how much does visual observation change the odds) from the certainties (areas, background rate). Now I can take that to several experts who will identify new mechanisms and tighten up my estimations on how big a deal the components are. In this way - even though the final % I come to would still be a bit arbitrary - it is starting to become a summary of what a large number of people think about the inputs to the problem and their relative magnitudes. Being able to communicate all that thinking with a single number is a miracle in its own way.
mr_toad 8y ago
It’s not that the election itself is subject to random uncertainty so much as the data that they are using to make the prediction is. The data might favour one candidate, but even assuming it is unbiased and representative, it is only a random sample, and there is a chance it could be randomly wrong.

seambot1 8y ago

I remember him saying before the election that the odds of Trump winning we're roughly the odds of a randomly picked day being a Saturday. Which always seemed pretty clear to me - Saturdays definitely happen, Trump could definitely happen. And then it did, and it was nothing like a Saturday, not at all.