5 ms·

In my opinion, 538 did some of the best analytics work in the 2016 election. They were the only ones who came anywhere close to property estimating the chances

In my opinion, 538 did some of the best analytics work in the 2016 election. They were the only ones who came anywhere close to property estimating the chances of the outcome that actually occurred, and they were one of only a couple models who took sufficiently seriously how unusual things were - particularly, the massive unpopularity of both candidates. I'm getting really irritated with people (in this thread and other) attacking 538 along partisan or personal lines for not making the same predictions they did personally. 538 made a model, not a poll; they are only ever as good as the polls they represent, and even then, they still managed to be better than quite a lot of public polling and commentary. If Hillary Clinton had won, the same kinds of people on the other side would be attacking 538 for being too bullish on Trump. The worst part is that a lot of this is predicated on the idea that giving something a 1 in 3 chance is the same as saying it'll never happen, when it's almost the opposite. There are some critical misunderstandings of statistics going on here, on all sides.

chimeracoder 10y ago
> The worst part is that a lot of this is predicated on the idea that giving something a 1 in 3 chance is the same as saying it'll never happen, when it's almost the opposite. There are some critical misunderstandings of statistics going on here, on all sides. On the morning of the election, FiveThirtyEight said that the chances of Trump winning were roughly equal to the chances of a random coin flip coming up heads twice in a row (25%). In that situation, a good statistician would certainly say that the expected outcome is for at least one of the coin flips to land on "tails" (75% chance). But if the coin lands on heads twice, you wouldn't say that "statistics is broken"[0] or that the statistician predicting the outcome was wrong. Similarly, let's say that you had a separate person predicting the probability of the double-heads coin flip to be 90%. If it does happen, that doesn't mean they made a good prediction, just because the outcome was correct. They predicted the correct outcome, but the degree of confidence they expressed in it was completely wrong. Predictions aren't just about getting the outcome right - they're also about expressing the right degree of confidence in the results. [0] Which is literally what some otherwise-respectable journalists complained after the election
- ivraatiems 10y ago
  You're totally right. The trouble is that when 538 was saying "wait, hold up, we're not sure," and in all their commentary being very clear that that 25% chance was a real, important possibility, people were taking it as, "oh, they think Hillary has it in the bag." People are taking their being rightfully less certain than anyone else using the same data as meaning they were just as wrong as anyone else, when the data itself was lacking. They weren't. (Of course, the degree to which the data was lacking and the reasons for that lack is another issue in and of itself.)
bendykstra 10y ago
> If Hillary Clinton had won, the same kinds of people on the other side would be attacking 538 for being too bullish on Trump. In fact, that very thing did happen in the week leading up to the election, so sure were some that Clinton would win. There was a raft of articles nitpicking the model, particularly the adjustments that it applied to polls and the level of uncertainty that it assumed.
- ivraatiems 10y ago
  > In fact, that very thing did happen in the week leading up to the election, so sure were some that Clinton would win. There was a raft of articles nitpicking the model, particularly the adjustments that it applied to polls and the level of uncertainty that it assumed. Ugh, yes, like that HuffPo guy attacking Nate Silver. He apologized later, at least.
Houshalter 10y ago
After the election, Silver said something to the effect of this. That given the polling data available, there is no way anyone could have favored Trump. Clinton was leading by a large amount in a large percentage of polls. You would have to apply some seriously creative statistics to make a model that favored Trump. Systemic polling error is a thing that happens. It's happened in previous elections. The polls are much more accurate than chance, but they aren't infallible. Particularly in this election, it's difficult to predict voter turnout from polls. 538 was one of the only models that took that into account. And as far as I know, they gave better odds to Trump than everyone else that tried to predict the election with statistical methods. Certainly they have a better track record than political pundits, which have never been better than chance.
- ghaff 10y ago
  Yep. And it's also worth remembering that FiveThirtyEight's thing is that they do analysis based on data as opposed to punditry. Yes, they apply corrections and otherwise filter the data sources. But some people seem to think that Silver should have gotten up a couple days before the election, state that he had a bad feeling about things, and dismiss the best forecast he could make based on some shaky polling data and gone with Trump based on gut feel. Sure, he would have ended up being correct. That would also run counter to the fundamental philosophy of the site.
  - lmg643 10y ago
    People forget that Silver's approach is basically weighting polls to figure out what the election will be, and he's the best "poll weigher" out there. But if the polls are systematically wrong, being the best at it is an unimpressive subtlety to the layperson. To the extent 2018 brings increasingly contentious elections, we will likely see polling to remain inaccurate due to voter unwillingness to state unpopular opinions. In that case, multi-input "big data" approaches may prevail. (Of course, it's also possible that 2016 is a "top" in terms of divisive rhetoric...only time can tell.)
    - remarkEon 10y ago
SixSigma 10y ago
Did the model write the headlines? June 16, 2015: Why Donald Trump Isn’t A Real Candidate, In One Chart July 16, 2015: Two Good Reasons Not To Take The Donald Trump ‘Surge’ Seriously July 20, 2015: Donald Trump Is The World’s Greatest Troll Aug. 6, 2015: Donald Trump’s Six Stages of Doom Aug. 11, 2015: Donald Trump Is Winning The Polls, And Losing The Nomination Nov. 23, 2015: Dear Media, Stop Freaking Out About Donald Trump’s Polls Donald Trump Comes Out Of Iowa Looking Like Pat Buchanan
- ivraatiems 10y ago
  No, but you're cherry picking very old articles, so I'm not terribly convinced. Commentary and articles as the election versus Clinton proceeded took him quite seriously.
  - phlakaton 10y ago
    To wit: 8 Nov: There's A Wide Range Of Outcomes, And Most of Them Come Up Clinton 6 Nov: How Much Did Comey Hurt Clinton's Chances? 6 Nov: Don't Ignore the Polls – Clinton Leads, But It's A Close Race 5 Nov: Why We Don't Know How Much Sexism Is Hurting Clinton's Campaign 4 Nov: National Polls Show Clinton's Lead Stabilizing – State Polls, Not So Much 4 Nov: Trump Is Just A Normal Polling Error Behind Clinton 3 Nov: Why Clinton's Position Is Worse Than Obama's 2 Nov: The How-Full-Is-This-Glass Edition 2 Nov: Trump's Chance Of Victory Has Doubled In The Past Two Weeks 1 Nov: Yes, Donald Trump Has A Path To Victory 1 Nov: On A Scale From 1 To 10, How Much Should Democrats Panic? 31 Oct: The Odds Of A Popular Vote-Electoral College Split Are Increasing 31 Oct: Comey Or Not, Trump Continues To Narrow Gap With Clinton 27 Oct: Don't Read Too Much Into Early Voting 26 Oct: Is The Presidential Race Tightening? 24 Oct: Why Our Models Are Much More Bullish Than Others On Trump You get the idea.
  - SixSigma 10y ago
    There was the now infamous Trump has as much chance of winning as Cubs sli.mg is being attacked so I can only post a link you can't see !! https://i.sli.mg/nElXDz.jpg https://i.sli.mg/nElXDz.jpg He was described as an "insider pundit" by the Clinton team https://wikileaks.org/podesta-emails/emailid/28878 https://wikileaks.org/podesta-emails/emailid/28878 From:brentbbi@webtv.net : ... I see Dan, Nate Silver and various other insider pundits saying what they are saying, I have been to enough rodeos to know that thoughts are being planted from the Obama-Clinton consultant class (not you, and you know who I mean). This is not helpful to Hillary, quite the contrary, ... So I don't consider him objective.
Nition 10y ago
I think sometimes people get confused about percent CHANCE versus actual estimated results. All those people need to do is go to the election forecast page, which is still up[1], and look at the percentages under "Popular vote". Or down lower under "How the forecast has changed", click POPULAR VOTE to see the percentages over time. For instance, the final pre-election estimate before the election gives: - CHANCE TO WIN of 71.4% Clinton vs. 28.6% Trump, around a 1 in 3.5 chance for Trump to win. - RESULTS ESTIMATE of 48.5% Clinton vs. 44.9% Trump, with a big overlapping margin of error. [1]https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/
throwaway287391 10y ago
> 538 made a model, not a poll; they are only ever as good as the polls they represent, and even then, they still managed to be better than quite a lot of public polling and commentary. I completely agree with almost all of your post, but want to defend the polls, even though you didn't explicitly blame the polls. FiveThirtyEight themselves consistently defend the polls even in the 2016 election. For example, they ran an article a few days before the election that Trump was only "a normal polling error" away from Clinton [1], and in their podcasts and articles since the election, have stated that in fact an average polling error was exactly what happened. The problem with a normal polling error in this case was that the election was close, and in particular, states that were critical for Clinton were very close. And FiveThirtyEight's model was the only one to really recognize the degree of uncertainty this really created by giving Trump a 25 or 33% chance. [1] https://fivethirtyeight.com/features/trump-is-just-a-normal-polling-error-behind-clinton/ https://fivethirtyeight.com/features/trump-is-just-a-normal-...

chimeracoder 10y ago

> The worst part is that a lot of this is predicated on the idea that giving something a 1 in 3 chance is the same as saying it'll never happen, when it's almost the opposite. There are some critical misunderstandings of statistics going on here, on all sides. On the morning of the election, FiveThirtyEight said that the chances of Trump winning were roughly equal to the chances of a random coin flip coming up heads twice in a row (25%). In that situation, a good statistician would certainly say that the expected outcome is for at least one of the coin flips to land on "tails" (75% chance). But if the coin lands on heads twice, you wouldn't say that "statistics is broken"[0] or that the statistician predicting the outcome was wrong. Similarly, let's say that you had a separate person predicting the probability of the double-heads coin flip to be 90%. If it does happen, that doesn't mean they made a good prediction, just because the outcome was correct. They predicted the correct outcome, but the degree of confidence they expressed in it was completely wrong. Predictions aren't just about getting the outcome right - they're also about expressing the right degree of confidence in the results. [0] Which is literally what some otherwise-respectable journalists complained after the election

ivraatiems 10y ago
You're totally right. The trouble is that when 538 was saying "wait, hold up, we're not sure," and in all their commentary being very clear that that 25% chance was a real, important possibility, people were taking it as, "oh, they think Hillary has it in the bag." People are taking their being rightfully less certain than anyone else using the same data as meaning they were just as wrong as anyone else, when the data itself was lacking. They weren't. (Of course, the degree to which the data was lacking and the reasons for that lack is another issue in and of itself.)

bendykstra 10y ago

> If Hillary Clinton had won, the same kinds of people on the other side would be attacking 538 for being too bullish on Trump. In fact, that very thing did happen in the week leading up to the election, so sure were some that Clinton would win. There was a raft of articles nitpicking the model, particularly the adjustments that it applied to polls and the level of uncertainty that it assumed.

ivraatiems 10y ago
> In fact, that very thing did happen in the week leading up to the election, so sure were some that Clinton would win. There was a raft of articles nitpicking the model, particularly the adjustments that it applied to polls and the level of uncertainty that it assumed. Ugh, yes, like that HuffPo guy attacking Nate Silver. He apologized later, at least.

Houshalter 10y ago

After the election, Silver said something to the effect of this. That given the polling data available, there is no way anyone could have favored Trump. Clinton was leading by a large amount in a large percentage of polls. You would have to apply some seriously creative statistics to make a model that favored Trump. Systemic polling error is a thing that happens. It's happened in previous elections. The polls are much more accurate than chance, but they aren't infallible. Particularly in this election, it's difficult to predict voter turnout from polls. 538 was one of the only models that took that into account. And as far as I know, they gave better odds to Trump than everyone else that tried to predict the election with statistical methods. Certainly they have a better track record than political pundits, which have never been better than chance.

ghaff 10y ago
Yep. And it's also worth remembering that FiveThirtyEight's thing is that they do analysis based on data as opposed to punditry. Yes, they apply corrections and otherwise filter the data sources. But some people seem to think that Silver should have gotten up a couple days before the election, state that he had a bad feeling about things, and dismiss the best forecast he could make based on some shaky polling data and gone with Trump based on gut feel. Sure, he would have ended up being correct. That would also run counter to the fundamental philosophy of the site.
- lmg643 10y ago
  People forget that Silver's approach is basically weighting polls to figure out what the election will be, and he's the best "poll weigher" out there. But if the polls are systematically wrong, being the best at it is an unimpressive subtlety to the layperson. To the extent 2018 brings increasingly contentious elections, we will likely see polling to remain inaccurate due to voter unwillingness to state unpopular opinions. In that case, multi-input "big data" approaches may prevail. (Of course, it's also possible that 2016 is a "top" in terms of divisive rhetoric...only time can tell.)
  - remarkEon 10y ago

SixSigma 10y ago

Did the model write the headlines? June 16, 2015: Why Donald Trump Isn’t A Real Candidate, In One Chart July 16, 2015: Two Good Reasons Not To Take The Donald Trump ‘Surge’ Seriously July 20, 2015: Donald Trump Is The World’s Greatest Troll Aug. 6, 2015: Donald Trump’s Six Stages of Doom Aug. 11, 2015: Donald Trump Is Winning The Polls, And Losing The Nomination Nov. 23, 2015: Dear Media, Stop Freaking Out About Donald Trump’s Polls Donald Trump Comes Out Of Iowa Looking Like Pat Buchanan

ivraatiems 10y ago
No, but you're cherry picking very old articles, so I'm not terribly convinced. Commentary and articles as the election versus Clinton proceeded took him quite seriously.
- phlakaton 10y ago
  To wit: 8 Nov: There's A Wide Range Of Outcomes, And Most of Them Come Up Clinton 6 Nov: How Much Did Comey Hurt Clinton's Chances? 6 Nov: Don't Ignore the Polls – Clinton Leads, But It's A Close Race 5 Nov: Why We Don't Know How Much Sexism Is Hurting Clinton's Campaign 4 Nov: National Polls Show Clinton's Lead Stabilizing – State Polls, Not So Much 4 Nov: Trump Is Just A Normal Polling Error Behind Clinton 3 Nov: Why Clinton's Position Is Worse Than Obama's 2 Nov: The How-Full-Is-This-Glass Edition 2 Nov: Trump's Chance Of Victory Has Doubled In The Past Two Weeks 1 Nov: Yes, Donald Trump Has A Path To Victory 1 Nov: On A Scale From 1 To 10, How Much Should Democrats Panic? 31 Oct: The Odds Of A Popular Vote-Electoral College Split Are Increasing 31 Oct: Comey Or Not, Trump Continues To Narrow Gap With Clinton 27 Oct: Don't Read Too Much Into Early Voting 26 Oct: Is The Presidential Race Tightening? 24 Oct: Why Our Models Are Much More Bullish Than Others On Trump You get the idea.
- SixSigma 10y ago
  There was the now infamous Trump has as much chance of winning as Cubs sli.mg is being attacked so I can only post a link you can't see !! https://i.sli.mg/nElXDz.jpg https://i.sli.mg/nElXDz.jpg He was described as an "insider pundit" by the Clinton team https://wikileaks.org/podesta-emails/emailid/28878 https://wikileaks.org/podesta-emails/emailid/28878 From:brentbbi@webtv.net : ... I see Dan, Nate Silver and various other insider pundits saying what they are saying, I have been to enough rodeos to know that thoughts are being planted from the Obama-Clinton consultant class (not you, and you know who I mean). This is not helpful to Hillary, quite the contrary, ... So I don't consider him objective.

Nition 10y ago

I think sometimes people get confused about percent CHANCE versus actual estimated results. All those people need to do is go to the election forecast page, which is still up[1], and look at the percentages under "Popular vote". Or down lower under "How the forecast has changed", click POPULAR VOTE to see the percentages over time. For instance, the final pre-election estimate before the election gives: - CHANCE TO WIN of 71.4% Clinton vs. 28.6% Trump, around a 1 in 3.5 chance for Trump to win. - RESULTS ESTIMATE of 48.5% Clinton vs. 44.9% Trump, with a big overlapping margin of error. [1]https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/

throwaway287391 10y ago

> 538 made a model, not a poll; they are only ever as good as the polls they represent, and even then, they still managed to be better than quite a lot of public polling and commentary. I completely agree with almost all of your post, but want to defend the polls, even though you didn't explicitly blame the polls. FiveThirtyEight themselves consistently defend the polls even in the 2016 election. For example, they ran an article a few days before the election that Trump was only "a normal polling error" away from Clinton [1], and in their podcasts and articles since the election, have stated that in fact an average polling error was exactly what happened. The problem with a normal polling error in this case was that the election was close, and in particular, states that were critical for Clinton were very close. And FiveThirtyEight's model was the only one to really recognize the degree of uncertainty this really created by giving Trump a 25 or 33% chance. [1] https://fivethirtyeight.com/features/trump-is-just-a-normal-polling-error-behind-clinton/ https://fivethirtyeight.com/features/trump-is-just-a-normal-...