7 ms·

This is such a damning list. At some point, reporting on polls when they have this bad of a track-record serves no journalistic purpose and just confuses the p

by OskarS 6y ago

This is such a damning list. At some point, reporting on polls when they have this bad of a track-record serves no journalistic purpose and just confuses the public. Like, the discussions people were having in the days before the election about Biden's strength barely resembles reality. I respect FiveThirtyEight and their work, and I do think they're intellectually honest and generally speaking good at their jobs. But they can only be as good as their sources, and when the sources are this terrible, they shouldn't be reported on this, and not given the statistical and scientific sheen of authenticity 538 gives them. Like, that site is starting to have a net negative influence on the world, and when that is the case with a journalistic institution, what are you even doing?

paulddraper 6y ago
> I respect FiveThirtyEight and their work, and I do think they're intellectually honest and generally speaking good at their jobs. But 538 chooses how to weight the polls. For example, they only gave Rasmussen (which was far closer on these) a C rating, preferring less accurate polls. FWIW, their articles clearly lean left [1], IDK if that affects their analysis/forecasts. [1] https://www.adfontesmedia.com/interactive-media-bias-chart-2/ https://www.adfontesmedia.com/interactive-media-bias-chart-2... similar to WaPo
- mewse 6y ago
  Boy, that rear view mirror makes criticism a whole bunch easier, doesn’t it? :)
systemvoltage 6y ago
One of the things that's annoying about 538 is that they don't admit they're wrong. The excuse they keep repeating is that "1 in 10 chance Trump can win. That's about the same chance as rain in LA, but it does rain in LA!". 538 seems to have forgotten in their condenscencion is that we are not look at you to explain the odds. We are looking at you to provide an accurate probability estimate. Their defense is completely ridiculous. I follow betting odds which had Biden/Trump at 60/40 split before the election day - what's the point of polling when you can look at people putting money on the line, let the incentives work for you.
- evgen 6y ago
  They provided the probability distribution. The fact that you can’t handle math and need some sort of absolute certainty for a future event is not 538’s problem.
  - systemvoltage 6y ago
    I am questioning - why even bother? It's not like we do elections every month to test out their probability distribution against empirical data. The distribution collapses into a binary outcome at the end. I have a dice. I claim the distribution is of equal outcome for each side. Well...we don't get to test the dice more than once. 1 sample size does not prove that the 538's predictions were right (or wrong). Thanks for assuming I can't do math, no way to argue with someone but I am actually pretty bad at it. :-)
    - evgen 6y ago
      Everyone is bad at probability and statistical distributions, not just you. The problem with modeling elections is that there are so few of them and the data is very noisy and until quite recently rather suspect. Let's not pretend that this was a normal election, either in the candidates running or in the manner in which the campaign and election was conducted. As to the question of why bother, it is because bad polling is better than no polling at all. Campaigns are now multi billion dollar enterprises managing tens of thousands of temporary employees for the creation of a product that will only be sold once and in 18+ months from when they start the process. Any data is better than nothing. The fact that the public has become obsessed with polls is probably due to the ongoing nationalization of politics.
  - listenallyall 6y ago
    it's not one single election -- it's consistent failure over multiple state elections, by large margins, all in the same direction -- which falls beyond any reasonable probability
    - roenxi 6y ago
chkaloon 6y ago
538 does make adjustments based on what they think are biases in individual polls. So they can't blame it all on their sources. They've taken on some responsibility to evaluate those sources and adjust their model accordingly. The polls had significant bias, and 538 failed to fully adjust for it.
ianai 6y ago
I am reminded of all the times in the last four years (really my entire adult life) where during policy discussions of any magnitude the ultimate “kill” for an idea is “but it’s only popular with $small_number of the vote” or “but poll after poll shows it as a losing position.” It’s used as a cudgel to push out things that would actually make people feel their government is doing things with impunity. Donald Trump’s approval rating rarely if ever dipped below ~43% (with the usual and huge error bars) and consequently republicans reported they couldn’t go against him ever because of the polls. “Spending political capital” is a concept where a ruling party takes unpopular actions/enacts unpopular law because it’ll only push their approvals toward their long run averages. I think this needs to be a serious topic of discussion. After this administration, “governing by polling firm” seems disingenuous at best and outright detrimental to all involved.

by OskarS 6y ago

paulddraper 6y ago

> I respect FiveThirtyEight and their work, and I do think they're intellectually honest and generally speaking good at their jobs. But 538 chooses how to weight the polls. For example, they only gave Rasmussen (which was far closer on these) a C rating, preferring less accurate polls. FWIW, their articles clearly lean left [1], IDK if that affects their analysis/forecasts. [1] https://www.adfontesmedia.com/interactive-media-bias-chart-2/ https://www.adfontesmedia.com/interactive-media-bias-chart-2... similar to WaPo

mewse 6y ago
Boy, that rear view mirror makes criticism a whole bunch easier, doesn’t it? :)

systemvoltage 6y ago

One of the things that's annoying about 538 is that they don't admit they're wrong. The excuse they keep repeating is that "1 in 10 chance Trump can win. That's about the same chance as rain in LA, but it does rain in LA!". 538 seems to have forgotten in their condenscencion is that we are not look at you to explain the odds. We are looking at you to provide an accurate probability estimate. Their defense is completely ridiculous. I follow betting odds which had Biden/Trump at 60/40 split before the election day - what's the point of polling when you can look at people putting money on the line, let the incentives work for you.

evgen 6y ago
They provided the probability distribution. The fact that you can’t handle math and need some sort of absolute certainty for a future event is not 538’s problem.
- systemvoltage 6y ago
  I am questioning - why even bother? It's not like we do elections every month to test out their probability distribution against empirical data. The distribution collapses into a binary outcome at the end. I have a dice. I claim the distribution is of equal outcome for each side. Well...we don't get to test the dice more than once. 1 sample size does not prove that the 538's predictions were right (or wrong). Thanks for assuming I can't do math, no way to argue with someone but I am actually pretty bad at it. :-)
  - evgen 6y ago
    Everyone is bad at probability and statistical distributions, not just you. The problem with modeling elections is that there are so few of them and the data is very noisy and until quite recently rather suspect. Let's not pretend that this was a normal election, either in the candidates running or in the manner in which the campaign and election was conducted. As to the question of why bother, it is because bad polling is better than no polling at all. Campaigns are now multi billion dollar enterprises managing tens of thousands of temporary employees for the creation of a product that will only be sold once and in 18+ months from when they start the process. Any data is better than nothing. The fact that the public has become obsessed with polls is probably due to the ongoing nationalization of politics.
- listenallyall 6y ago
  it's not one single election -- it's consistent failure over multiple state elections, by large margins, all in the same direction -- which falls beyond any reasonable probability
  - roenxi 6y ago

chkaloon 6y ago

538 does make adjustments based on what they think are biases in individual polls. So they can't blame it all on their sources. They've taken on some responsibility to evaluate those sources and adjust their model accordingly. The polls had significant bias, and 538 failed to fully adjust for it.

ianai 6y ago

I am reminded of all the times in the last four years (really my entire adult life) where during policy discussions of any magnitude the ultimate “kill” for an idea is “but it’s only popular with $small_number of the vote” or “but poll after poll shows it as a losing position.” It’s used as a cudgel to push out things that would actually make people feel their government is doing things with impunity. Donald Trump’s approval rating rarely if ever dipped below ~43% (with the usual and huge error bars) and consequently republicans reported they couldn’t go against him ever because of the polls. “Spending political capital” is a concept where a ruling party takes unpopular actions/enacts unpopular law because it’ll only push their approvals toward their long run averages. I think this needs to be a serious topic of discussion. After this administration, “governing by polling firm” seems disingenuous at best and outright detrimental to all involved.