8 ms·
As someone who was super interested in the 538-style of election coverage in 2008, I've kind of fallen "out of love" so to speak with election models and foreca
by qqtt 2y ago
As someone who was super interested in the 538-style of election coverage in 2008, I've kind of fallen "out of love" so to speak with election models and forecasting in general. I'm not really convinced about what it adds to the conversation around elections. We can all look at various polls and get an assessment of who is generally ahead. Weighted polling aggregators and forecasting models just collect all these polls and spit out some data. It's easy to hand wave and think some new information is being revealed, but ultimately it is just a "garbage in garbage out" situation - you are entering polls as input, some hand waving is going on, and you get some forecast as a result.
I think part of my cynicism comes in the wake of the 2016 election, in which the forecast rightfully counted some scenarios in which either candidate could win, upon which conclusion of the model was basically "the result fits in with the forecast, because either candidate could have won according to the model" - in which case I personally concluded, if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here? We can simply look at polls and understand who is generally ahead, and not be any better or worse off.
- fitsumbelay 2y agoagreed on all points
- smt88 2y agoThe purpose of statistical coverage of elections is to tell campaigns what their strengths and weaknesses are and to entertain the rest of us. Since we don't run campaigns, the only value is entertainment. But that's not a negligible value for a lot of people.
- ks2048 2y agoI agree it's mainly entertainment, but it also affects a lot of people "involved" in campaigns who aren't "running" them - mainly people donating their money or time.
- sdwr 2y agoI kinda get your point - statistics suck the air out of the room. If regular people are talking about swing state poll margins of error instead of the actual issues, something's gone wrong. 538 democrasized the numbers that were the domain of political whizzes. I don't know if that's a good thing.
- wslack 2y agoI think aggregators are useful to tell us what actually moved voters in a campaign - the swings are visible even if baseline error is not.
- bee_rider 2y agoEh, this seems to just promote armchair quarterbacking. What moved voters is an issue for the campaigns to track. We should listen to what the campaigns say about what we care about.
- maigret 2y agoWell swing states are the issues, aren’t they? I’m happy to live in a place where it isn’t a small fraction of individuals deciding the fate of the country.
- tbrownaw 2y agoBut how else will you know the correct emotional tone to use when complaining / gloating about the upcoming election with friends? Or know how seriously to take various prepper activities like stocking up on Twinkies in case the wrong team wins?
- Uehreka 2y agoI find the modeling super useful, many conventional media outlets still don’t properly communicate probabilities to their audience. For instance, I vividly remember the following exchange between Nate and some news anchor in during one of the 2016 conventions: Anchor: So Nate, you say Trump has a 25% chance of winning, can you tell us exactly what that means? Nate: Sure. So imagine I flipped two quarters, and they both came up heads. In that scenario, Trump wins. Anchor: (shocked) Wait but that’s… that’s a thing that actually happens! You’re saying Trump has an actual chance of winning? Nate: Well I’d rather be Hillary than Trump right now, but yes, people shouldn’t be that surprised if Trump wins, his chances aren’t insignificant. I remember people in October still saying that Nate had to be wrong, that there was just no way Trump could win. There was even a growing market for what I would now call “cope forecasts” that “unskewed” the results to show that, really, Hillary had a 99% chance of winning, just like you knew she did (all of these people looked extremely foolish after the election was over). I also feel like good models provide valuable pushback against media narratives that try to characterize the “closeness” of a race. In 2016, people wanted to hear that Trump had no chance of winning, but Nate/538 correctly pushed back that the race was actually pretty close and both candidates had a good chance. And he did the opposite in 2012: Pundits wanted to cast Obama and Romney as being neck-and-neck (which makes for a more exciting story) and Nate had the stats to push back that actually the race was not very close at all. If Romney had won in 2012, Nate would’ve had to eat crow, but Romney didn’t win. Nate and 538 also do senate races, which are super valuable if you’re figuring out which candidates to donate money to. Often there are Democratic candidates in totally doomed races against Republicans I really don’t like, and the data helps me look at those situations and go “yeah I hate Lindsey Graham, but his challenger has no chance, I’m going to donate to the milquetoast Nevada senator whose race is on a knife’s edge instead”. I could probably just look up polls, but the way Nate/538 process the polls into results with error bars and probabilities makes it a lot easier to reason about.
- MostlyStable 2y agoI agree with basically this whole comment, but the sad/ironic/whatever part of it is though even though Nate Silver was doing the thing you are describing ("actually, Trump has a real shot of winning"), afterwards he got constantly dinged (either by people incorrectly conflating him with the 99% models, or by people who just didn't actual listen to him) for "getting it wrong" because he "only" gave trump a 30ish % chance of winning. I still come across it every once in a while and it probably has the highest ratio of level of infuriating-ness to low-value of the stakes of the opinion of just about any political opinion I can think of.
- nozzlegear 2y agoI largely agree with your points. Election modelers and forecasters really don't add much to the conversation after 2016, despite their attempts and even purported success at correcting their models and mistakes. The only election forecasting model that I take seriously these days is my own vibes based forecasting. I have enjoyed the meta-drama around forecasting and modeling that pops up every election season though. It's hard to beat "[Nate Silver] doesn't have the faintest idea how to turn the keys," or "I ran 80,000 simulations." ¹ https://knowyourmeme.com/memes/he-doesnt-have-the-faintest-idea-how-to-turn-the-keys https://knowyourmeme.com/memes/he-doesnt-have-the-faintest-i... ² https://knowyourmeme.com/memes/i-ran-80000-simulations https://knowyourmeme.com/memes/i-ran-80000-simulations
- deleted 2y ago[deleted]
- mgfist 2y agoBut that's because elections these days are incredibly close. It's like being upset that the best statistical answer to "who will win a coin toss" is "well it's 50/50".
- genewitch 2y agoAre you sure about that? 89% of counties turned red. Looking at 1.5% total difference is missing the forest for the trees. Trump got all swing states. To quote: "this is absolutely a mandate." https://youtu.be/zG3n2IeaTPA https://youtu.be/zG3n2IeaTPA edit: facts cannot be insolent. The youtube link is a guest on the view named Stephen something, from this week, saying the words that i typed into the comment box. Not clicking it is doing yourself a disservice; as it is "source cited." Don't get mad at me for relaying this information.
- roenxi 2y agoSure. And what was the probability of something like that happening? About a coin flip. It is an election using First-Past-the-Post counting. A 50:50 probability doesn't mean the final count will be close, it (usually, anyway) means there is evens odds which side is about to win decisively. The county results are expected to correlate, as are the swing states in all likelihood.
- genewitch 2y agothe probability of 89% of blue counties (in 2020) switching to red counties is a 50-50 chance? can you show your work?
- roenxi 2y agohttps://projects.fivethirtyeight.com/2024-election-forecast/ https://projects.fivethirtyeight.com/2024-election-forecast/ You'll note that the distribution mode for a Trump win was 312 electoral votes; which is what he got. They do a good forecast - it suggests there was a 50-50 chance Trump wins, and if he wins the most likely outcome is ... exactly what happened. If it was a Harris win the best bet is she would have gotten 319 electoral votes with high correlation in the swing states too.
- alisonatwork 2y ago538 in its final form was not about predicting election outcomes, it was about the business and science of polling, and contrasting factual data with the way people perceived related issues. It was an interesting outlet that helped illuminate the data side of political science, and at its best also provided some insight into the disconnect between how the general public thinks about a topic versus what is actually happening. Disney killing 538 is broadly a loss for political journalism in the US, imo, because most other American media is more interested in sensationalism and hyping imaginary culture war issues, i.e. exacerbating exactly the disconnect with reality that 538 was trying to combat with its more evidence-based reporting. From my perspective the only place still doing this kind of work outside of niche, single-topic outlets like SCOTUSblog is ProPublica, and even they don't tend to be as politics-centric as 538 was. So I definitely will miss the site, and the pod. I don't have the stomach for most other American media.
- wasabi991011 2y agoRespectfully, I disagree on both points. > We can all look at various polls and get an assessment of who is generally ahead. I probably could, but there's a lot of polls to look through and I don't really want to spend the time. Much rather have someone else do it for me. > if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here? Isn't this hypothesis testing? If you have a weighted coin and a guess as to which side is heavier and by how much, you're going to need multiple flips to see if you are right. And it doesn't even really make sense to talk about how right/wrong you are about a single flip, only on the aggregate. It's possible someone has already compiled FiveThirtyEight's results to get some aggregate accuracy, I haven't checked. If they have and he's wrong on average and that's what you are referencing, my apologies.
- JeremyNT 2y agoThe trick here is that if the election is close enough that you'd actually want/need multiple pollsters aggregated, the aggregators will indicate high uncertainty. If it's enough of a blowout for the aggregators to indicate low uncertainty, then the individual polls are going to be showing a large gap. An aggregator saying "foo has a 65% chance of winning" may seem like it's providing more information than a single historically reliable poll (say Reuters/Ipsos) stating "foo is up by 2 points but there's a 3 point margin of error" - but isn't it just an illusion? High quality pollsters very seldom deviate very much. And even if you grant that the aggregator is closer to being "right" than any single pollster, is that difference actually meaningful enough to impact any real world behaviors? Would you do anything differently with a 50% chance of victory versus a 70% chance? I've honestly come to think of them as entertainment, with no real value.
- tqi 2y agoI know the folks at 538 meant well, but I think the ultimate impact of their work was to accelerate the politics as entertainment, team sports-ification of elections, to our nations detriment.
- CalRobert 2y agoI find it useful in the general way that having corroborating sources for election data is useful. If the election is reasonably close to models, exit polls, etc. then it is more likely to have been run fairly.
- mikepurvis 2y agoAs a resident of Kitchener ON, I'm privileged to live in one of just two Canadian ridings represented by the Green Party— I worked on Mike Morrice's first campaign in 2019, and I remember being frustrated trying to talk to voters on doorsteps and having to explain over and over that the purported "polls" on 338 (Canadian knockoff 538) showing us at 3-4% support in the riding were based on projecting forward previous results and adjusting slightly for national polling trends. Sure enough, we ended up coming in at a whopping 26% in 2019, and in 2021 won the seat with 33%. Certainly the win was in part because the incumbent was embroiled in a last minute scandal, but I truly believe the polling aggregators have a huge suppressing effect on breakout candidates— without that effect it's possible we could have taken the earlier election too. Now that seat is "safely" Green, it's been twice affirmed with huge wins for a separate Green candidate at the provincial level: https://en.wikipedia.org/wiki/Kitchener_Centre_(federal_electoral_district) https://en.wikipedia.org/wiki/Kitchener_Centre_(federal_elec... https://en.wikipedia.org/wiki/Kitchener_Centre_(provincial_electoral_district) https://en.wikipedia.org/wiki/Kitchener_Centre_(provincial_e... I expect this year's federal election will deliver another 40-50% result for Morrice, as he's very popular locally, but there's 338 again showing a big upswing for the Liberals in Kitchener Centre, when almost certainly there is no such thing, it's all just hallucinated from national polling: https://338canada.com/35048e.htm https://338canada.com/35048e.htm
- bnralt 2y agoLike the Rationalist's "Bayesian priors," the election models were a remnant of the "big data" hype from a decade and a half ago. This article is a decent overview for anyone who forgot about it[1]. Like with many hype cycles, there was something actually important underneath the surface (useful statistical modeling), but then people with a poor understanding of the limitations ran wild thinking it could do things far beyond its capabilities (in this case, the degree to which one could use statistics to predict the future). Industry gave up on the more extreme claims fairly quickly because it wasn't able to produce. But it lingered on in other places where there was less direct feedback or it was telling people what they wanted it to hear. To add to this, it became obvious that many of the leaders in this "field" were people who believed they had an expertise that was far beyond their actual capabilities. Nate Silver ended up accusing much of the polling industry of fraud recently, because he wasn't able to do basic statistical math[2]. [1] https://slate.com/technology/2017/10/what-happened-to-big-data.html https://slate.com/technology/2017/10/what-happened-to-big-da... [2] https://x.com/JustinWolfers/status/1853302476406993315 https://x.com/JustinWolfers/status/1853302476406993315
- Gimpei 2y agoI disagree. Before 538 people were still offering lots of election predictions and it was much much worse, because it was based entirely on hunches and vibes. Silvers rates the pollsters and provides confidence intervals far better than a simple average of polls does. I’d much rather read his forecasts than any number of bloviating opeds.
- JumpCrisscross 2y agoWould note that Silver owned the election model 538 ran on. When he left, he took it with him. The recent election forecasts 538 put out were not Nate Silver’s. (In my opinion, his were more accurate. More importantly, his commentary was more informative on the model’s shortcomings and insights.)
- finnthehuman 2y ago>if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here? When the "upset" candidate wins, it will have been statistically likely. Yes. What they're adding are error bars in the collective consciousness. Ironically enough, the hype pop of 538 was actually driven by people misinterpreting the stats and using them to feel vindicated in their support of Obama. The comedown was finding out that 25% is still 1-in-4 odds. 538 should be killing the horse race coverage by doing the most sober version of it. But horse race is big business. Therefore: boo 538, booooooo, they're harshing the vibe of my favorite reality show.