49 ms·

One pollster’s explanation for why the polls got it wrong

179 points by satchet 6y ago

zepto 6y ago
This presumably also affects polls on things that measure support for individual issues related to trust in society - for example Gun control.
- SV_BubbleTime 6y ago
  You can also look at previous votes on initiatives as semi-accurate polling. Michael Bloomberg / Everytown for Gun Safety / Moms Demand Action / The Trace / Mayors Against Illegal Guns which are all the same entity repeatedly pushed out the statistic that "90% of guns owners want to end private gun sales" of course this is totally misleading, but... When it came a vote in Washington state, after outspending the NRA over 10:1 on Initiative-594 they received a 60/40 win. I would put that 90% a lot farther down, because everyone in WA voted on this, not just gun owners. So, its it 10%, 30%, 60%? IDK, but I know it's not 90%.
  - zepto 6y ago
    Previous votes generally aren’t semi-accurate. For example ask people to vote on ‘universal background checks’ alone, you’ll probably get a majority voting in favor. If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it.
    - SV_BubbleTime 6y ago
      Yea, I can agree to that. Let's say then that we can't make determinations, but can tell when something is just definitely not true, like 90% of gun owners definitely do not support ending private sales. >If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it. Man, whole separate topic right there! People not realizing what gun laws say vs mean is so common it seems like it's by design.
0x53 6y ago
One of the biggest issues I think is that the polling is still done by just randomly calling people. I think I agree that there is a large skew introduced by the simple fact of only reaching people willing to answer a call from an unknown number.
- deleted 6y ago
  [deleted]
- legitster 6y ago
  This isn't strictly true though, there are any number of polling methods that pollsters use these days including online polling and essentially pre-approved call databases. One of the examples in this article was an in-person poll with 70% response rate. It's probably more accurate to say it's harder to effectively weigh or average responses in a setting where there are 4 or 5 different channels you are collecting responses from.
  - samth 6y ago
    The GSS spends (according to Shor) $1000 per response. It's not at all similar to the "polls that were wrong" in the election.
  - pmiller2 6y ago
    That’s interesting, considering I would be far more likely to dodge an in-person poll than any other type. If I see anyone with a clipboard asking people if they’re registered voters, I give a wide berth.
  - flomo 6y ago
    Public radio once had a guest describing their online polling methodology - it was running "Win a Prize" type banner ads. So probably not very representative either.
- koolba 6y ago
  Even the method of contact would skew toward people who have a means for the method of contact or location. Cell phones might include less seniors. Questioning people walking into Whole Foods is going to give you a different set than people walking into Cracker Barrel. If the source set of names and numbers is truly representative, I think random calling could work if you make sure to include "did not response" as a line item. Then you might get something like: > 45% Biden, 39% Trump, 1% Jorgensen, %15 ??? With the no-response removed you get a skewed: > 52% Biden, 45% Trump, 1.1% Jorgensen
jxramos 6y ago
I'm curious where polls actually work, are there any retrospectives polling agencies conduct and post online for scrutiny and constructive feedback? It'll probably help to enumerate all the premises that polling begins with and question whether those premises are in fact the case.
- dgr582systems 6y ago
  Exactly this. Where is the verification? Predictions don’t count for much unless you check that the expectations line up with reality.
  - dragonwriter 6y ago
    IIRC, 538 has available both the polling data and predictions for all of their political forecast series (not just Presidential.)
  - danhor 6y ago
    538 has some infos about this here: https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/
- wombatmobile 6y ago
  > I'm curious where polls actually work If your definition of "work" is that a poll accurately predicts the outcome of the election, take a step back. Polls don't do that. I'm not just referring to inaccuracies that are introduced by non-representedness of the sample, although that's clearly a factor as well. Polls invite respondents to indicate how they would vote "if an election were held today". But the election isn't being held today, and the respondents know it. Polls are an invitation for respondents to send a message to politicians before the poll, to influence the politicians' offerings. Respondents know that, and use the opportunity that way. So what are polls? Polls are their own little universe, with their own characteristics, dynamics, and effects. What they aren't is a crystal ball.
- handol 6y ago
  They work well for what seems to be their intended purpose. Providing fodder for political entertainment to the 24hr news networks.
oconnore 6y ago
Could you select a random group of individuals up front and then bid on their collective participation in a poll (say, beyond 95%, you all get $10)? You might have to pick a few groups and bid between them.
- jeremysalwen 6y ago
  In the future if you are chosen to be polled, you will be stalked, harassed, and bribed until you respond.
  - mabbo 6y ago
    I imagine that will be a huge boost to candidates who promise to pass laws that make aggressive polling against the law.
- lacker 6y ago
  The problem is that some of the people in your random group won’t answer the phone at all, to negotiate with you on whether they’ll answer a survey.
- dillondoyle 6y ago
  Yes. A lot of the digital surveys you see do this. E.g. they enroll people in mechanical turk. They claim it's similar to a Nielsen group but I don't buy it. We do truly random surveys delivered in mobile ads. Can't ask nearly as many questions but I think it's one of the best ways to currently get a large truly representative sample
ericb 6y ago
Has anyone done a study that maps voting machine software used vs. "polling error" in key states?
- daxfohl 6y ago
  Before 2016, polls were much more accurate. And paper ballots were the norm. So there's that.
  - bpodgursky 6y ago
    ? A huge percent of ballots this year were mail-in, and all paper. And the polling error was worse in 2020 that 2016.
    - ericb 6y ago
      If I were going to try an influence an election via voting machine software, I would affect only some percentage of votes so it wouldn't be too obvious--if there was also mail in ballots, it wouldn't matter. I might also target only key locations--those on the margins where it wouldn't be suspicious. If I were to do this in favor of the Republican candidate, you'd probably find that mail-in votes even in non-pandemic years skewed more Democratic. You'd also find that polls seemed to indicate that Republican voters were "shy" and it keeps messing up the polls.
    - baldeagle 6y ago
      Were the mail in returns against the poll? Or only the in-persons? I know in my area of Texas, the mail-in returns were more Dem than the polls, but the in-persons were much more Rep. Could it be that one side focused on in-person voting and self selected into in person votes.... perhaps. Could it be hacking or other fraud, not very likely. Such accusations need a lot of support of proof, because faith in democracy depends on faith in elections. I don't think we're seeing enough proof anywhere. (I would look to any place that has a mandatory recount where ballots are hand counted). I don't think any Texas races were that close though.
    - dragonwriter 6y ago
kanox 6y ago
TL;DR: People lie to pollsters.
- twelfthnight 6y ago
  > People tell the truth, when you ask them who they’re voting for. They really do, on average. The reason why the polls are wrong is because the people who were answering these surveys were the wrong people. The takeaway from the article is not that people lie, but just that people who are more trusting (according to the GSS survey) are more likely to take surveys and are also more likely to vote Democratic.
  - shinjitsu 6y ago
    Especially if those polled fear for their jobs if they answer the 'wrong way' in the poll and others find out about it: https://twitter.com/AOC/status/1324807776510595078 https://twitter.com/AOC/status/1324807776510595078 -> https://www.trumpaccountability.net/ https://www.trumpaccountability.net/
    - tshaddox 6y ago
      How could that tweet possibly have anything to do with polls? Any remotely scientific poll would surely be anonymous. That has nothing to do with people willingly tweeting their political views and then attempt to cover them up or expect them to be forgotten in the future.
      - parliament32 6y ago
        If you get a call from an unknown caller asking "hey who're you voting for", how do you know if it's a proper scientific pollster or your employer's brother-in-law?
      - thu2111 6y ago
        They're based on calling phone numbers, which are tied to identities, and the President has access to the NSA databases that correlate everyone's identities with their phones, IP addresses, etc. If people in Congress or the President wanted to, they could absolutely find out who answered in polls that they'd vote for a candidate.
gnusty_gnurc 6y ago
It seems obvious to me that there was massive bias in media towards Biden, and they all happily accepted the narrative that Biden would win in a landslide and dismissed anyone saying otherwise as know-nothings. IMO that's tantamount to election interference, if there's near unanimous media coverage for the entire election season of inaccurate polling that assumes a landslide Biden victory. I didn't vote for Biden or Trump, so I'm not really interested in defending either camp here. I just think the media and big tech are so demonstrably contemptible and partial.
- bryanlarsen 6y ago
  If the media should be giving equal status and weight to all the parties, what about the Libertarians and the Greens?
  - wavefunction 6y ago
    And the Communist and Constitution Parties.
  - gnusty_gnurc 6y ago
    > If the media should be giving equal status and weight to all the parties I'm not saying they need to give them equal status, necessarily - I'm saying that the employees of these institutions exist in a bubble mostly cause of class&geographic reasons. The point is that all of these positions "Biden will naturally win in a landslide per polling" are tacitly accepted and challenges are dismissed as hearsay. There needs to be some challenge/discourse against the elite orthodoxy.
    - bryanlarsen 6y ago
      I didn't see any of the elite media saying "Biden will naturally win in a landslide per polling". They were saying that in 2016. In 2020 much more common was "the polls are showing a landslide, but I don't trust the polls completely any more". If you are saying that the polls were deliberately biased, I disagree strongly. It's a really freaking hard job if only 1-3% of the people you try to poll don't hang up the phone on you. That the polling error was under 10% was incredible in that scenario. The media is biased. The polls had an error. Two very different things.
rootusrootus 6y ago
Maybe it doesn't really matter much. I mean, sure, we need some amount of polling because it informs decisions such as who gets to participate in televised debates, etc. But other than that, what does the general public get out of polling? It's a spectator sport. In the end, all that matters is the votes, and people should really be encouraged to remember this. I'd think 2016 and 2020 would be a healthy reminder of that.
- ffggvv 6y ago
  i could picture polling having an effect of suppressing the vote such as in 2016. where one side doesnt feel the need to vote since their candidate is the clear favorite.
- smnrchrds 6y ago
  > what does the general public get out of polling? The laypeople, not much. But for politically active people, not to mention political parties, polls are the best tool to determine where to do what. If you are way ahead in one race but neck and neck in another, it makes sense for parties and activists to focus their efforts and donations to the competitive race. There is never enough resources to focus on every race equally and to the fullest extent. Decisions need to be made. Polls are the best tool to make informed decisions.
  - hundchenkatze 6y ago
    > polls are the best tool to determine where to do what Seems like that may not be the case anymore.
  - AnimalMuppet 6y ago
    OK, but when some random pollster calls me, what do I get out of it if the parties can allocate their resources more effectively? That's not anywhere on my list of concerns. It lets you campaign more effectively? Great, go bother someone else.
    - smnrchrds 6y ago
jonplackett 6y ago
When and where was the last poll that predicted an election accurately? Polling is starting to feel more like a weather forecast - presented with absolute certainty, only right about half the time.
- TechBro8615 6y ago
  Trafalgar group did a pretty good job this election. They were also lambasted by Democratic media and called wackos and nutjobs. Meanwhile WaPo put its name on a poll that had Biden +17 in WI.
  - tptacek 6y ago
    No, they didn't. They were off by 4 in AZ (and called it for Trump), by 5 in Georgia (again for Trump), 6 in Michigan (again), 8 in Wisconsin (and again), and 3 in PA (which they, again, called for Trump). They were also off by 7 in Minneapolis (which they correctly called for Biden) and 6 in Ohio (which they correctly called for Trump). There was a narrative on e-day that Trafalgar had done a good job, because Democratic votes were disproportionately counted later in the week. But if you'd relied on Trafalgar predictions to allocate resources in an election, you'd have gotten depantsed.
    - tgb 6y ago
      Moreover the Trafalgar head of polling went on Fox a day before the election claiming that there would be 4% points of systematic election fraud happening in Pennsylvania. So "nutjob" and "wacko" would be underselling them. https://www.mediaite.com/tv/trafalgar-pollster-robert-cahaly-makes-baseless-claim-trump-will-win-pennsylvania-but-they-will-steal-it-with-voter-fraud/ https://www.mediaite.com/tv/trafalgar-pollster-robert-cahaly...
- mindlar 6y ago
  Some of the local ones have done well. The Des Moines, Iowa poll has been pretty close. The national polling firms are only getting airplay if they give the "right" result, the few groups that have been pretty accurate in 2016, 2018, and 2020 got very little media coverage due to not being close to the high profile ones and not having the results that CNN, ABC, NBC, etc. wanted to broadcast. That coupled with comments from some prominent Democratic politicians that they want to make lists of Trump supporters presumably to punish them for their actions, has a significant chilling effect on people being willing to vocalize their support for Trump to non-friendly or unknown people.
x87678r 6y ago
I think no one really knows what the different parties represent any more. Most people I know voted Republican were low-wage working people, where my highly paid educated friends voted Democrat. Isn't it supposed to be other way around? No wonder everyone is confused.
- TechBro8615 6y ago
  It’s pretty obvious from election maps and donor data that the democrats are now the party of coastal/urban elites, despite what they may advertise.
  - paulryanrogers 6y ago
    This discounts tens of millions of Democrats living inland who are obscured by maps depending on the electoral college. It seems to me the pattern is that where population density increases the people tend to be more politically leftward-leaning.
    - ttesttom 6y ago
      There was an interesting chart from the economist today about that, which showed a strong correlation. The denser the area the more democratic. https://twitter.com/gelliottmorris/status/1326250981131169794?s=20 https://twitter.com/gelliottmorris/status/132625098113116979...
  - watwut 6y ago
    Majority of Americans lives on the coast and in cities. They can't be all elite.
  - triceratops 6y ago
    You're saying the majority of voters this year were elite? That's a whole lotta elites...kinda waters down the definition no?
  - dragonwriter 6y ago
    > It’s pretty obvious from election maps and donor data that the democrats are now the party of coastal/urban elites Urban populations, sure. Not particularly coastal aside from the fact that there's more urban population on the coasts, as county-level maps show pretty clearly. “Elites” are more dubious. Sure, Democrats and Republicans (counting leaners) are closer to parity for voters with less than a college degree, with college grads and even moreso postgraduate degree holders strongly favoring Democrats. OTOH, Blacks, who are overwhelming not elites, overwhelming are (and even more overwhelming lean) Democratic, and the same is true predominantly (though less overwhelmingly) of Hispanics. So, yes, to the urban part, not particularly independently true of the “elite” or “coastal” part.
wbeckler 6y ago
Here's another explanation, pre-election, of the gap, the found that Trump voters are more likely to lie to pollsters: https://www.cloudresearch.com/resources/blog/election-2020-poll-respondent-honesty/ https://www.cloudresearch.com/resources/blog/election-2020-p... These pollsters agreed with this phenomenon, arrived at from a totally different direction: https://www.politico.com/news/magazine/2020/10/29/2020-polls-trump-biden-prediction-accurate-2016-433619 https://www.politico.com/news/magazine/2020/10/29/2020-polls...
auganov 6y ago
As bad as election polls are, at least you get to find out the truth eventually. A lot of polling has to do with domains where there's no easy way of establishing ground truth. But ironically it would appear to me people tend to trust these polls more. While most seem to be skeptical about election polling, they might share exit polls on, say, demographics, without questioning their accuracy.
gridspy 6y ago
TL;DR: Distrustful people don't talk to strangers. They vote differently too.
- bra-ket 6y ago
  it's a bit hard to trust mainstream media that have been lying to you and called you 'racist' for 4 years.
  - mring33621 6y ago
    Voting for racists makes you a racist.
    - bra-ket 6y ago
      define 'racist' and what makes you think Trump is a 'racist'
      - 0xffff2 6y ago
        "Shithole countries" and a refusing to denounce white supremacy make for an easy start.
        bra-ket 6y ago
        He denounced white supremacy many times, you are misinformed. What ‘shithole countries’ have to do with racism
        colinmhayes 6y ago
        Denouncing white supremacists after the entire country says you have to isn't really the same. Anyone with critical thinking skills can see that the number of times he has refused to denounce white supremacists or called them good people makes him a white supremacist.
bra-ket 6y ago
>People who don’t answer polls .. tend to have low levels of trust in other people no, we just don't trust your propaganda machine
beervirus 6y ago
Here's a good illustration of just how one-sided the polling error was: https://twitter.com/thomasjwood/status/1325103908319064065 https://twitter.com/thomasjwood/status/1325103908319064065 The polls hugely and consistently underestimated the Republican votes.
- jackcosgrove 6y ago
  I would be interested to see these errors plotted against state population density.
johnnyg 6y ago
The book Thinking in Bets points out that predictions are given with a % of certainty. If I say X party will win at Y% and Z party wins, it doesn't mean the model is broken.
- jerf 6y ago
  On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!" Given two major elections in a row where the results were essentially out of the error bars, it is perfectly reasonable to be dropping your confidence on pollsters very hard. Once was perhaps understandable, but twice is getting into "right .01% of the time" territory.
  - johnnyg 6y ago
    I hadn't considered the "unfalsifiable" aspect of my comment. I think you make a good point. At a poker table you get a high number of events to check against but a presidential election is only once every 4 years. I wonder how accurate the local elections are vs the national? If neither is accurate, then that's a strong case for your point of view and a change in methodology.
  - cwkoss 6y ago
    > On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!" I wonder how many people still think Nate Silver has a credible perspective: he's spun the above strategy into a job in unscientific punditry.
- rayiner 6y ago
  But elections aren’t single events, they are 50 different events. 5% of the time the results in Michigan will be outside the error bars. But if the results are outside the error bars in Michigan, Iowa, Wisconsin, Minnesota, etc., and the polls all got it wrong in the same direction for each (overestimated Biden’s support) that’s actually statistically quite unlikely.
bryanlarsen 6y ago
Another pollsters opinion: Nate Cohn says the story is about Latino voters who are a notoriously hard demographic to poll. Source "The Daily" podcast.
- SV_BubbleTime 6y ago
  Did Nate Cohn's predictions this election pan out? “The final polls more or less comport with how we already viewed the race. Mr. Biden ends the race up by more than eight points nationwide — the largest lead a candidate has held in the final polls since Bill Clinton in 1996. He’s up by at least five points in states worth more than 270 electoral votes, the number needed to win. Beyond that, he’s got at least a nominal lead in states worth 350 electoral votes, and he’s just a 2012 polling error away from a sweeping landslide of more than 400 electoral votes.” https://www.nytimes.com/live/2020/presidential-polls-trump-biden/assessing-the-last-polls-before-the-election https://www.nytimes.com/live/2020/presidential-polls-trump-b... Cohn's polling leading up to the election: Tuesday Nevada Biden +6 Wednesday Michigan Biden +8 Thursday N.C. Biden +3 Sunday Arizona Biden +6 Sunday Florida Biden +3 Sunday Pennsylvania Biden +6 Sunday Wisconsin Biden +11 Which of these happened? Biden underperformed compared to Clinton in most pivot counties and bellweather predictions (which were 90% wrong this year, wow), and most counties in swing states entirely with the except of greaty overperforming in the densest liberal areas, in some of those cases even doing better than Obama as crazy as that seems. Biden underperformed with hispanics and black from exceptions and likewise Trump did better with them than expected overall. And no matter what if you think someone the day of the election saying "Biden Landside" was possible, well... You do you, but me personally, I'm going to go to someone else for opinions because that was such an exaggeration of reality that it seems like it had to be "for effect".
  - bryanlarsen 6y ago
    Somebody who got it wrong and actually admitted he got it wrong is often a credible source on why he got it wrong.
    - SV_BubbleTime 6y ago
evo_9 6y ago
Who replies honestly to political question to random strangers? I know I don't. What's the benefit of letting anyone know where you stand politically at this point? None. It's much more likely - no matter which side you're for - that the person asking could have an adverse reaction. Things are just way too polarized now, not sure how that can change, but that's how it is. I mean outside friends/family I simply take a neutral, non-committal stance on almost every hot-button issue these days. And honestly even that's not 100% true, I avoid getting into deep political conversations with most of my family, I simply don't want to cause further rifts in our already frail relations over all this stuff. There is no middle ground.
- dillondoyle 6y ago
  Research has confirmed that people don't lie to pollsters in any meaningful way when asked their head to head preference. It's mentioned in the last graph of the article. The crux of this article is rather than 'shy trump voters' there is simply a large amount of these Trump voters who will refuse to take a survey flat out.
  - colinmhayes 6y ago
    Isn't trump voters refusing to take polls what everyone means when they say "shy trump voters"?
    - PeterisP 6y ago
      As far as I understand, the traditional sense of "shy voter" is to describe your poll numbers (i.e. the people who did take the survey) which answered "undecided" when they were shy about their choice. This also goes hand in hand with things like observed differences in answers of in-person surveys when the person doing the survey is/isn't from the same demographic as the person being surveyed. A disproportional amount of people (after controlling for race/gender/income/education/etc) simply refusing to answer the survey has similar but conceptually different effect; one that (as the OP claims - perhaps they're mistaken) is new and had not been a significant factor before.
hashtagmarkup 6y ago
This isn't new information. Whoever is funding pollsters should know better, and pollsters themselves should be ashamed for taking the money. https://en.wikipedia.org/wiki/Bradley_effect https://en.wikipedia.org/wiki/Bradley_effect
rayiner 6y ago
> I reached out to the smartest polling guy I know: David Shor, an independent data analyst who’s a veteran of the Obama presidential campaigns who formerly operated a massive web-based survey at Civis Analytics before leaving earlier this year Before being fired for pointing out that violent protests alienate people: https://nymag.com/intelligencer/2020/06/case-for-liberalism-tom-cotton-new-york-times-james-bennet.html https://nymag.com/intelligencer/2020/06/case-for-liberalism-... Schor’s firing is a tremendous example of how free speech values make people better at their job, and attacks on free speech values can jeopardize organizations and political parties. It’s conventional wisdom now that failure to condemn violent protests earlier cost Democrats votes, including among minority groups. This was quite apparent from polling this summer. (By June, YouGov polls showed that Latinos were more worried about the “breakdown of law and order” than about “systemic racism.”) Creating a culture that makes people afraid to have dispassionate, context-appropriate discussions about what the data was saying made it harder for people to do their jobs effectively. Obviously what kinds of facts are relevant to an organization’s mission differs greatly depending on the organization! In many contexts Schor’s comments wouldn’t have been relevant and may have been better left unsaid to foster workplace harmony. But in a political organization, obviously some heated issues will come up and people need to be able to have honest discussions about them.
- fastball 6y ago
  "How did Trump get so many votes?" The same people that attacked Schor have undoubtedly also asked themselves this question, without an ounce of self-awareness. Except instead of recognizing their own culpability, the conclusion most such people are coming to seems to be "well, I guess nearly half of all Americans are bad people".
  - flowerlad 6y ago
    Not bad people, but gullible people. Anyone that believes QAnon conspiracy theories such as Hillary Clinton, Bill Gates, and Oprah Winfrey are raping and drinking the blood of kidnapped children in pursuit of the “fountain of youth” is gullible.
ttamslam 6y ago
For no particular reason, I feel very strongly that you could achieve much more accurate results by asking people to estimate what _they_ think the final margin will be then average those results. I'm inclined to think that community members know their community well enough that given a large enough sample, this average will trend towards the true result. https://en.wikipedia.org/wiki/Wisdom_of_the_crowd https://en.wikipedia.org/wiki/Wisdom_of_the_crowd edit: A quick google search found: https://www.researchgate.net/publication/289826354_Predicting_elections_A_%27Wisdom_of_Crowds%27_approach https://www.researchgate.net/publication/289826354_Predictin...
- mgraczyk 6y ago
  You may find this specific technique interesting, in which you try to find the "expert minority" in the crowd, under the assumption that people who can better predict the crowd's belief can also better predict the true outcome: https://en.wikipedia.org/wiki/Surprisingly_popular https://en.wikipedia.org/wiki/Surprisingly_popular
  - ttamslam 6y ago
    This is very interesting, thank you!
- dillondoyle 6y ago
  There's actually some good research on that. Another source - i think i've read a couple journal articles on this https://academic.oup.com/poq/article/78/S1/204/1836551#xref-corresp-1-1 https://academic.oup.com/poq/article/78/S1/204/1836551#xref-...
- jedberg 6y ago
  A good way to get a good prediction is to ask how they think their neighbor will vote. A person who is ashamed of their selection will be much more willing to tell you what it is if they think you're asking about their neighbors.
koiz 6y ago
People lie to pollsters or straight up won't admit who they voted for. Pollsters seem convinced otherwise.
AntiImperialist 6y ago
From election strategy point of view, you are in a much stronger position if you poll much higher, for various reasons: - uncaring and undecided voters, if they had to vote, are likely to vote for the winners, because being on the winning side makes them feel better (which may have decided this election) - if you lose the election, people are more likely to believe that the other side cheated if you were projected a clear winner by the polls, so you can bring people on the streets or run fraudulent lawsuits to deter the opponents and still get a lot of support So, strategically, it pays to poll higher regardless of how things really are. Given how a lot of media organizations are against Trump, it's not unlikely that these polling organizations were too... and they helped the opponents.
Animats 6y ago
Decades ago, phone polling got over a 50% response rate. Now it's under 10%. So there's at least a bias towards people who are willing to accept unsolicited calls.
- istjohn 6y ago
  The article says the response rate is just 1%.
  - Animats 6y ago
    That must be newer data. At 1%, the data is mostly noise.
sien 6y ago
Tim Harford has an excellent article on why the polls were wrong: https://timharford.com/2020/11/why-the-polls-got-it-wrong/ https://timharford.com/2020/11/why-the-polls-got-it-wrong/ The big quote is this: "The pre-election numbers suggest that in a typical poll with 500 positive responses, 255 went for Mr Biden and 243 for Mr Trump. But typical response rates are 5 in 100 — often lower, says Andrew Gelman, a statistician and prominent election modeller. He says that only around 1 in 100 people respond to opinion polls. So now picture 10,000 people, 255 who called their vote for Mr Biden, 243 for Mr Trump and 9,500 who never responded. How confident are we feeling now? " Harford is the host of the excellent More or Less podcast that looks at the validity of statistics in the news.
drilldrive 6y ago
Frankly, this has an obvious answer: The people who answer polls are not representative of the population at large. Yet this was known decently enough since 2016. But more at the heart of the issue here is pollsters are providing a service not to inform but to persuade (both the public and donors); internal-polling is actually what matters. And instead of reformulating the service with more accurate questions for total populace (who will your neighbor vote for?), the pollsters doubled down on the standard methods and made (roughly doubly!) a larger polling error than 2016. I mean the Maine senate race was on average ~15 points off the final result[0]. Frankly this was the race for the pollsters to prove themselves that they can self-correct, as they have demonstrated to be unable to do this, it is clear they just should be abolished. [0] https://www.realclearpolitics.com/epolls/2020/senate/me/maine_senate_collins_vs_gideon-6928.html# https://www.realclearpolitics.com/epolls/2020/senate/me/main...
- noncoml 6y ago
  > it is clear they just should be abolished I am not sure I understand what you mean. Abolished how?
  - drilldrive 6y ago
    What I mean to say is polling as a major industry for political campaigns is done after this election. The current people running these public polls have no place of confidence anymore: donors should not take them of value, and public opinion should ignore polling. They had their chance this time around for survival as an industry but roundly threw that away.
asdff 6y ago
A: sampling bias.
gshubert17 6y ago
I've kept a spreadsheet to compare the last polling estimates from fivethirtyeight.com with the actual election results. Although there may be 8 or 9 million votes left to count, the percentages are probably good enough to make some comparisons. What I saw was an average polling error of 4.9 percentage points. The actual results for Trump were 4.9 points higher than the polls. And the errors were greater in states that had higher percentages of votes for Trump. For example, North Dakota: The last polls showed 37.9% for Biden and 59.6% for Trump, a 21.7% difference. The latest popular vote tally from CNN shows 31.8% to 65.1%, which are 33.3% apart. That's an 11.6% error, much larger than expected sampling errors of 3-4%. Shor's explanation was interesting. Whether the explanation is right or not, I can see the effect Shor is describing.
mastazi 6y ago
This is my personal experience: over the course of my life, I have held both conservative positions and more progressive ones (depending on the topic, and in different periods of my life). I have always been much more willing to discuss openly those positions that I would describe as progressive. This is despite the fact that over the years I have been living in more than a handful of different countries, each one with different cultural norms. So I think there must be some sort of general tendency, where conservative opinions tend to be disclosed less frequently. I don't know why that is the case, and I would like to hear if anyone has an answer.
- deleted 6y ago
  [deleted]
- fiblye 6y ago
  I think it also depends on the audience and group you're with. If you're moving around a lot, you might end up in expat/international/diverse groups. Most people who move to several countries aren't moving to villages 4 hours from a city and speaking and blending in with the locals and really hearing what rural people think. Growing up in a small town and moving between small towns across the US every year or so, I never had exposure to "liberals" until I started using the internet as a teen. Until then, everyone I was ever around talked about them like they were bogeymen. In the country I live in now, I was initially exposed to much more "progressive" opinions from people. Now I'm seen as less "that foreign guy" and more of a local, and I hear what people would normally discuss with their friends and family. It's quite different.
  - mastazi 6y ago
    It's true that context counts, I can see your point since, similar to your experience, I also come from a rural area and I lived in a tiny town for some years. But I also feel like I have a "default setting" for situations where I don't really know who I'm talking to, and in those situations, what I said above still applies.
- apta 6y ago
  > I don't know why that is the case, and I would like to hear if anyone has an answer. Cancel culture and SJW mobs.
jungletime 6y ago
Dr.SHIVA Ayyadurai, MIT PhD. "Inventor of Email" Analyzed the results in Michigan and came to the conclusion that 69k votes were taken from Trump and given to Biden. His reasoning here: https://www.youtube.com/watch?v=Ztu5Y5obWPk&feature=youtu.be https://www.youtube.com/watch?v=Ztu5Y5obWPk&feature=youtu.be
- ZephyrBlu 6y ago
  Do you have a TL;DW? I would watch it all if it was a bit shorter or there were timestamps, but 1hr10min is a lot.
  - jungletime 6y ago
    From my understanding: He found that the more a republican a county is, the more republicans voted for Gop candidates, and against Trump. But only after the first 20% or so of the vote was in. In the first 20% of the vote Trump was doing better than the Gop candidates on the ticket. Comes to the conclusion that an algorithm is syphoning votes from Trump to Biden. In a predictable linear fashion after the first 20% of the vote is counted. And flips more votes in more republican counties in an almost perfect linear way. It doesn't make sense because Trump was polling better than the Gop candidates before the election, and was popular with republican voters. The tallying software has a weighted vote feature. Where votes can be weighted for example 2:1 for one candidate over the other. Thinks that this was kicked in after 20% of the vote was in. Which would produce a linear drop that the graphs show.
- rjkennedy98 6y ago
  That was not convincing at all and had some really glaring errors. Dr Shiva somehow confuses percent of overall voters with percent of Republicans to make it seem like there is a conspiracy going on. He says that margin for Trump should be a flat line so that if there 5% republican all-ballot voters it should be off by the same margin in Trump vote as a precinct with 80%. However if 20% of republicans flip that would mean 1% flipped vs 16% in those examples which is a line with a slope. Literally flipping those graphs would make them look the same for Biden as for Trump. All it shows is basically a kind of regression to the mean. Precincts with high republican or democratic all-ballot voting weren’t as republican or democratic as they seem. There could be many good reasons for that such as that voters that go against the way their communities vote tend to not be all-ballot voters. Also, Dr. Shiva is not an independent voice. He ran in Massachusetts as a Republican.
DecoPerson 6y ago
“The kind of people who answer polls are really weird, and it’s ruining polling.” This is true everywhere. I’ve been involved polling in for a popular AAA game, a small retail business, school events, online forums, and more; they all fail to account for the “who chooses to answer” bias (is there a name for this?). I speculate that the “average 4.7 million viewers“ rating Firefly received when it aired was totally off. The Fox executives (who clearly didn’t care for the show, if you read the full story) used this as ammunition to have it cancelled. (see https://en.m.wikipedia.org/wiki/Nielsen_ratings https://en.m.wikipedia.org/wiki/Nielsen_ratings for criticisms of the rating system used at the time). The people who would take the time to answer a poll (or connect a device to their TV for some cash) are a small subset of people who will skew your data in unintuitive ways. If more business people understood this, our world would be a better place.
- euank 6y ago
  > is there a name for this bias? I think it's response bias. https://en.wikipedia.org/wiki/Response_bias https://en.wikipedia.org/wiki/Response_bias
  - devinplatt 6y ago
    I think actually op is referring to https://en.wikipedia.org/wiki/Participation_bias https://en.wikipedia.org/wiki/Participation_bias: "Participation bias or non-response bias is a phenomenon in which the results of elections, studies, polls, etc. become non-representative because the participants disproportionately possess certain traits which affect the outcome." Confusing terminology, but I guess "response bias" is about a false response by the participant to a question, and "non-response bias" is about whether the selected participant responded (participated) at all.
saeranv 6y ago
To me, this article highlights how much of a moving target weighting polls, and political prediction is. Shor here speaks about how Democrats get oversampled as both in 2016, and 2020 due to different levels of engagement and trust over the election: >> It used to be that once you control for age and race and gender and education, that people who trusted their neighbors basically voted the same as people who didn’t trust their neighbors. But then, starting in 2016, suddenly that shifted. If you look at white people without college education, high-trust non-college whites tended toward [Democrats], and low-trust non-college whites heavily turned against us. In 2016, we were polling this high-trust electorate, so we overestimated Clinton. These low-trust people still vote, even if they’re not answering these phone surveys. Is this shift partially because Trump and then QAnon have shifted Trump-voters trust in institutions - thus the voters that don't respond to pollsters are shifting as the Republican party becomes more radicalized?
RickJWagner 6y ago
My theory is that it's the High School Effect. Democrats have highly effective social messaging, led by Hollywood and professional athletes. They constantly put out messaging, making dissenters appear 'uncool'. Sad to say, adults are actually swayed by such childish forces. How else to explain the constant error to the left in polls?
ab3rC1te 6y ago
It's called cheating, fake news, and whatever else you want to deem it as. That's why the Postal service got a 10 billion stimulus package. Just like COVID is so dangerous that it jumps 26 ft in cold air, punches your plexiglass cause it wants to attack you, and put that mask on, it can't go through that either. Even though it's air borne and been air borne since it was released. Just look up and see what's being sprayed in the atmosphere to rain down COVID 21 on you. When will people stop being idiots to the bullshit lies that they feed you.
EGreg 6y ago
This is ridiculous! Here I am telling people that Republicans are distrustful of mainstream media, science, colleges etc. and that pollsters must know what they're doing, getting representative samples is the best we can do etc. And then I found out what I suspected is true. The most trivial thing of some people being overrepresented in a poll by the mere fact that they agree to take one, apparently, was somehow TOO MUCH FOR THE POLLSTERS TO HANDLE. Are you kidding me? The methodology should involve getting truly REPRESENTATIVE and RANDOM samples. While each one is going to be biased in some way, the bias should be independent of political affiliation. For example, stopping people on a street. Yes it excludes people who don't go out on the street, or people who won't stop. Then you are supposed to note how many people refused to take the poll. And then you are supposed to measure the correlation between that and their beliefs. If you really can't do this, then have a sign outside the voting areas, as people exit, with Donald Trump or Biden or JoJo etc. and have them sign up for polling in the next four years. This way at least you can make sure going in that your population is balanced as of four years ago.
mmcgaha 6y ago
Everyone knows Republicans are racist homophobic xenophobic bigoted nazis. Why would they want to admit that to some random over the phone? The rhetoric that we are using to describe the other party is not necessarily helping our cause in the long run. First it is pushing folks with legitimate disagreement away from public discourse. Second it is creating a good-guy/bad-guy scenario that is the equivalent of tribalism. At this point I can pick between a political talk show or professional wrestling and get about the same quality of content.
- bArray 6y ago
  You should really edit your comment's first paragraph to make it clear that it's not your opinion. I agree that there is currently an atmosphere where people don't necessarily feel comfortable talking about their political leanings among strangers.
  - bananadrama 6y ago
    > You should really edit your comment's first paragraph to make it clear that it's not your opinion. It's clear from context.
    - nikitaga 6y ago
      My eyes roll faster than context arrives. Deliberately appearing as a troll until the crucial context arrives to reverse the entire meaning is a shit narrative device when discussing serious issues. Save this stuff for stupid memes.
      - JeremyNT 6y ago
        > Deliberately appearing as a troll until the crucial context arrives to reverse the entire meaning is a shit narrative device when discussing serious issues. Save this stuff for stupid memes. Even with the context it's obviously a troll, because the poster spends the entire first graph building a straw man. The underlying implication is that Republicans are somehow persecuted against by a nameless group who believes all of these negative attributes apply, but no evidence is presented for this.
martyvis 6y ago
"The voter approaches their table, and gives their name and party affiliation. The party affiliation is used as a signal that something fishy might be going on if the ratio of R to D is very different from the votes recorded." This sounds so weird to an Aussie. For most people when you enter the polling place, you get bombarded by party supporters with their "how to vote" leaflets. Most people take all or none to be polite and not indicate your preference. Inside you give your name and it is ruled through on the roll printout and you are handed your ballot papers. Once you vote in your little cardboard both you slot the paper in the box. You go outside, hand back the how to vote leaflets, and chat with people after you have bought your "democracy sausage" ( a hot dog bought from the local school or fire service fundraiser). No one really cares how you voted (and 95% of people eligible vote)
- AtlasBarfed 6y ago
  Do you guys have voting days off?
  - izacus 6y ago
    I'm guessing the vote is on Sundays?
AtlasBarfed 6y ago
If pollsters from a long and academic history are getting this wrong, and it basically is a two-sided yes/no question, what validity is there in twitter/facebook mining sentiment analysis?
tchock23 6y ago
One potential solution is to pay people to participate in polls. Pollsters don’t do this because it is prohibitively expensive and cuts into their consulting margins, but it can reduce sampling bias (it can also introduce a bias if not used effectively).
- Kylekramer 6y ago
  Online pollsters mostly "pay" their respondents, and did just as poorly as phone pollsters this go around.
disown 6y ago
I wonder what Facebook or Google's "poll" numbers would have been if they decided to run one? Given their access to search, texts, social media posts, emails, tracking, etc along with their profile of a large segment of the population, I'm guessing their polling data would be the most accurate.
- axaxs 6y ago
  Facebook already determines your political leaning, somehow. At least a few years ago you could see it for yourself, not sure if that still exists. I'm sure without a shadow of a doubt Google makes the same determination. So they probably don't need to 'run' anything special here.
burlesona 6y ago
> Qualitative research doesn’t solve the problem of one group of people being really, really excited to share their opinions, while another group isn’t. As long as that bias exists, it’ll percolate down to whatever you do. Pretty good TLDR from the end of the article.
diebeforei485 6y ago
Why were the errors only in one direction?
- _fizz_buzz_ 6y ago
  Because they are systematic errors e.g. Latinos are hard to poll. Latino vote went more in the direction of Trump then previously. So, every pollster is running into that issue.
ospohngellert 6y ago
Does anyone else reject the premise of the article? I understand that polling definitely missed big in some areas, but isn't that normal? Polling is really hard. You have to make a lot of assumptions, adjust for demographics, predict whether a voter will turn out, while using a sample of usually ~1000 people. Of course some polls are going to be wrong. I don't think the polling was generally wrong though. Polling averages had Biden +8 or so, when all votes are counted he'll likely be +5, which is within margin of error. Polls were extremely accurate in Georgia, where conventional wisdom would say Biden shouldn't win. I get that most of the polling error benefited Trump, and the industry needs to reflect on why that happened and how to prevent underestimating republicans again in the future, but I don't yet buy the narratives that "the polls got it wrong."
deleted 6y ago
[deleted]
chmod600 6y ago
Is there any solution? Perhaps using more proxies and indicators rather than direct questions?
scoot_718 6y ago
or voter suppression, which polling doesn't take into account.

179 points by satchet 6y ago

This presumably also affects polls on things that measure support for individual issues related to trust in society - for example Gun control.

SV_BubbleTime 6y ago
You can also look at previous votes on initiatives as semi-accurate polling. Michael Bloomberg / Everytown for Gun Safety / Moms Demand Action / The Trace / Mayors Against Illegal Guns which are all the same entity repeatedly pushed out the statistic that "90% of guns owners want to end private gun sales" of course this is totally misleading, but... When it came a vote in Washington state, after outspending the NRA over 10:1 on Initiative-594 they received a 60/40 win. I would put that 90% a lot farther down, because everyone in WA voted on this, not just gun owners. So, its it 10%, 30%, 60%? IDK, but I know it's not 90%.
- zepto 6y ago
  Previous votes generally aren’t semi-accurate. For example ask people to vote on ‘universal background checks’ alone, you’ll probably get a majority voting in favor. If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it.
  - SV_BubbleTime 6y ago
    Yea, I can agree to that. Let's say then that we can't make determinations, but can tell when something is just definitely not true, like 90% of gun owners definitely do not support ending private sales. >If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it. Man, whole separate topic right there! People not realizing what gun laws say vs mean is so common it seems like it's by design.

0x53 6y ago

One of the biggest issues I think is that the polling is still done by just randomly calling people. I think I agree that there is a large skew introduced by the simple fact of only reaching people willing to answer a call from an unknown number.

deleted 6y ago
[deleted]
legitster 6y ago
This isn't strictly true though, there are any number of polling methods that pollsters use these days including online polling and essentially pre-approved call databases. One of the examples in this article was an in-person poll with 70% response rate. It's probably more accurate to say it's harder to effectively weigh or average responses in a setting where there are 4 or 5 different channels you are collecting responses from.
- samth 6y ago
  The GSS spends (according to Shor) $1000 per response. It's not at all similar to the "polls that were wrong" in the election.
- pmiller2 6y ago
  That’s interesting, considering I would be far more likely to dodge an in-person poll than any other type. If I see anyone with a clipboard asking people if they’re registered voters, I give a wide berth.
- flomo 6y ago
  Public radio once had a guest describing their online polling methodology - it was running "Win a Prize" type banner ads. So probably not very representative either.
koolba 6y ago
Even the method of contact would skew toward people who have a means for the method of contact or location. Cell phones might include less seniors. Questioning people walking into Whole Foods is going to give you a different set than people walking into Cracker Barrel. If the source set of names and numbers is truly representative, I think random calling could work if you make sure to include "did not response" as a line item. Then you might get something like: > 45% Biden, 39% Trump, 1% Jorgensen, %15 ??? With the no-response removed you get a skewed: > 52% Biden, 45% Trump, 1.1% Jorgensen

jxramos 6y ago

I'm curious where polls actually work, are there any retrospectives polling agencies conduct and post online for scrutiny and constructive feedback? It'll probably help to enumerate all the premises that polling begins with and question whether those premises are in fact the case.

dgr582systems 6y ago
Exactly this. Where is the verification? Predictions don’t count for much unless you check that the expectations line up with reality.
- dragonwriter 6y ago
  IIRC, 538 has available both the polling data and predictions for all of their political forecast series (not just Presidential.)
- danhor 6y ago
  538 has some infos about this here: https://projects.fivethirtyeight.com/checking-our-work/ https://projects.fivethirtyeight.com/checking-our-work/
wombatmobile 6y ago
> I'm curious where polls actually work If your definition of "work" is that a poll accurately predicts the outcome of the election, take a step back. Polls don't do that. I'm not just referring to inaccuracies that are introduced by non-representedness of the sample, although that's clearly a factor as well. Polls invite respondents to indicate how they would vote "if an election were held today". But the election isn't being held today, and the respondents know it. Polls are an invitation for respondents to send a message to politicians before the poll, to influence the politicians' offerings. Respondents know that, and use the opportunity that way. So what are polls? Polls are their own little universe, with their own characteristics, dynamics, and effects. What they aren't is a crystal ball.
handol 6y ago
They work well for what seems to be their intended purpose. Providing fodder for political entertainment to the 24hr news networks.

oconnore 6y ago

Could you select a random group of individuals up front and then bid on their collective participation in a poll (say, beyond 95%, you all get $10)? You might have to pick a few groups and bid between them.

jeremysalwen 6y ago
In the future if you are chosen to be polled, you will be stalked, harassed, and bribed until you respond.
- mabbo 6y ago
  I imagine that will be a huge boost to candidates who promise to pass laws that make aggressive polling against the law.
lacker 6y ago
The problem is that some of the people in your random group won’t answer the phone at all, to negotiate with you on whether they’ll answer a survey.
dillondoyle 6y ago
Yes. A lot of the digital surveys you see do this. E.g. they enroll people in mechanical turk. They claim it's similar to a Nielsen group but I don't buy it. We do truly random surveys delivered in mobile ads. Can't ask nearly as many questions but I think it's one of the best ways to currently get a large truly representative sample

ericb 6y ago

Has anyone done a study that maps voting machine software used vs. "polling error" in key states?

daxfohl 6y ago
Before 2016, polls were much more accurate. And paper ballots were the norm. So there's that.
- bpodgursky 6y ago
  ? A huge percent of ballots this year were mail-in, and all paper. And the polling error was worse in 2020 that 2016.
  - ericb 6y ago
    If I were going to try an influence an election via voting machine software, I would affect only some percentage of votes so it wouldn't be too obvious--if there was also mail in ballots, it wouldn't matter. I might also target only key locations--those on the margins where it wouldn't be suspicious. If I were to do this in favor of the Republican candidate, you'd probably find that mail-in votes even in non-pandemic years skewed more Democratic. You'd also find that polls seemed to indicate that Republican voters were "shy" and it keeps messing up the polls.
  - baldeagle 6y ago
    Were the mail in returns against the poll? Or only the in-persons? I know in my area of Texas, the mail-in returns were more Dem than the polls, but the in-persons were much more Rep. Could it be that one side focused on in-person voting and self selected into in person votes.... perhaps. Could it be hacking or other fraud, not very likely. Such accusations need a lot of support of proof, because faith in democracy depends on faith in elections. I don't think we're seeing enough proof anywhere. (I would look to any place that has a mandatory recount where ballots are hand counted). I don't think any Texas races were that close though.
  - dragonwriter 6y ago

kanox 6y ago

TL;DR: People lie to pollsters.

twelfthnight 6y ago
> People tell the truth, when you ask them who they’re voting for. They really do, on average. The reason why the polls are wrong is because the people who were answering these surveys were the wrong people. The takeaway from the article is not that people lie, but just that people who are more trusting (according to the GSS survey) are more likely to take surveys and are also more likely to vote Democratic.
- shinjitsu 6y ago
  Especially if those polled fear for their jobs if they answer the 'wrong way' in the poll and others find out about it: https://twitter.com/AOC/status/1324807776510595078 https://twitter.com/AOC/status/1324807776510595078 -> https://www.trumpaccountability.net/ https://www.trumpaccountability.net/
  - tshaddox 6y ago
    How could that tweet possibly have anything to do with polls? Any remotely scientific poll would surely be anonymous. That has nothing to do with people willingly tweeting their political views and then attempt to cover them up or expect them to be forgotten in the future.
    - parliament32 6y ago
      If you get a call from an unknown caller asking "hey who're you voting for", how do you know if it's a proper scientific pollster or your employer's brother-in-law?
    - thu2111 6y ago
      They're based on calling phone numbers, which are tied to identities, and the President has access to the NSA databases that correlate everyone's identities with their phones, IP addresses, etc. If people in Congress or the President wanted to, they could absolutely find out who answered in polls that they'd vote for a candidate.

gnusty_gnurc 6y ago

It seems obvious to me that there was massive bias in media towards Biden, and they all happily accepted the narrative that Biden would win in a landslide and dismissed anyone saying otherwise as know-nothings. IMO that's tantamount to election interference, if there's near unanimous media coverage for the entire election season of inaccurate polling that assumes a landslide Biden victory. I didn't vote for Biden or Trump, so I'm not really interested in defending either camp here. I just think the media and big tech are so demonstrably contemptible and partial.

bryanlarsen 6y ago
If the media should be giving equal status and weight to all the parties, what about the Libertarians and the Greens?
- wavefunction 6y ago
  And the Communist and Constitution Parties.
- gnusty_gnurc 6y ago
  > If the media should be giving equal status and weight to all the parties I'm not saying they need to give them equal status, necessarily - I'm saying that the employees of these institutions exist in a bubble mostly cause of class&geographic reasons. The point is that all of these positions "Biden will naturally win in a landslide per polling" are tacitly accepted and challenges are dismissed as hearsay. There needs to be some challenge/discourse against the elite orthodoxy.
  - bryanlarsen 6y ago
    I didn't see any of the elite media saying "Biden will naturally win in a landslide per polling". They were saying that in 2016. In 2020 much more common was "the polls are showing a landslide, but I don't trust the polls completely any more". If you are saying that the polls were deliberately biased, I disagree strongly. It's a really freaking hard job if only 1-3% of the people you try to poll don't hang up the phone on you. That the polling error was under 10% was incredible in that scenario. The media is biased. The polls had an error. Two very different things.

rootusrootus 6y ago

Maybe it doesn't really matter much. I mean, sure, we need some amount of polling because it informs decisions such as who gets to participate in televised debates, etc. But other than that, what does the general public get out of polling? It's a spectator sport. In the end, all that matters is the votes, and people should really be encouraged to remember this. I'd think 2016 and 2020 would be a healthy reminder of that.

ffggvv 6y ago
i could picture polling having an effect of suppressing the vote such as in 2016. where one side doesnt feel the need to vote since their candidate is the clear favorite.
smnrchrds 6y ago
> what does the general public get out of polling? The laypeople, not much. But for politically active people, not to mention political parties, polls are the best tool to determine where to do what. If you are way ahead in one race but neck and neck in another, it makes sense for parties and activists to focus their efforts and donations to the competitive race. There is never enough resources to focus on every race equally and to the fullest extent. Decisions need to be made. Polls are the best tool to make informed decisions.
- hundchenkatze 6y ago
  > polls are the best tool to determine where to do what Seems like that may not be the case anymore.
- AnimalMuppet 6y ago
  OK, but when some random pollster calls me, what do I get out of it if the parties can allocate their resources more effectively? That's not anywhere on my list of concerns. It lets you campaign more effectively? Great, go bother someone else.
  - smnrchrds 6y ago

jonplackett 6y ago

When and where was the last poll that predicted an election accurately? Polling is starting to feel more like a weather forecast - presented with absolute certainty, only right about half the time.

TechBro8615 6y ago
Trafalgar group did a pretty good job this election. They were also lambasted by Democratic media and called wackos and nutjobs. Meanwhile WaPo put its name on a poll that had Biden +17 in WI.
- tptacek 6y ago
  No, they didn't. They were off by 4 in AZ (and called it for Trump), by 5 in Georgia (again for Trump), 6 in Michigan (again), 8 in Wisconsin (and again), and 3 in PA (which they, again, called for Trump). They were also off by 7 in Minneapolis (which they correctly called for Biden) and 6 in Ohio (which they correctly called for Trump). There was a narrative on e-day that Trafalgar had done a good job, because Democratic votes were disproportionately counted later in the week. But if you'd relied on Trafalgar predictions to allocate resources in an election, you'd have gotten depantsed.
  - tgb 6y ago
    Moreover the Trafalgar head of polling went on Fox a day before the election claiming that there would be 4% points of systematic election fraud happening in Pennsylvania. So "nutjob" and "wacko" would be underselling them. https://www.mediaite.com/tv/trafalgar-pollster-robert-cahaly-makes-baseless-claim-trump-will-win-pennsylvania-but-they-will-steal-it-with-voter-fraud/ https://www.mediaite.com/tv/trafalgar-pollster-robert-cahaly...
mindlar 6y ago
Some of the local ones have done well. The Des Moines, Iowa poll has been pretty close. The national polling firms are only getting airplay if they give the "right" result, the few groups that have been pretty accurate in 2016, 2018, and 2020 got very little media coverage due to not being close to the high profile ones and not having the results that CNN, ABC, NBC, etc. wanted to broadcast. That coupled with comments from some prominent Democratic politicians that they want to make lists of Trump supporters presumably to punish them for their actions, has a significant chilling effect on people being willing to vocalize their support for Trump to non-friendly or unknown people.

x87678r 6y ago

I think no one really knows what the different parties represent any more. Most people I know voted Republican were low-wage working people, where my highly paid educated friends voted Democrat. Isn't it supposed to be other way around? No wonder everyone is confused.

TechBro8615 6y ago
It’s pretty obvious from election maps and donor data that the democrats are now the party of coastal/urban elites, despite what they may advertise.
- paulryanrogers 6y ago
  This discounts tens of millions of Democrats living inland who are obscured by maps depending on the electoral college. It seems to me the pattern is that where population density increases the people tend to be more politically leftward-leaning.
  - ttesttom 6y ago
    There was an interesting chart from the economist today about that, which showed a strong correlation. The denser the area the more democratic. https://twitter.com/gelliottmorris/status/1326250981131169794?s=20 https://twitter.com/gelliottmorris/status/132625098113116979...
- watwut 6y ago
  Majority of Americans lives on the coast and in cities. They can't be all elite.
- triceratops 6y ago
  You're saying the majority of voters this year were elite? That's a whole lotta elites...kinda waters down the definition no?
- dragonwriter 6y ago
  > It’s pretty obvious from election maps and donor data that the democrats are now the party of coastal/urban elites Urban populations, sure. Not particularly coastal aside from the fact that there's more urban population on the coasts, as county-level maps show pretty clearly. “Elites” are more dubious. Sure, Democrats and Republicans (counting leaners) are closer to parity for voters with less than a college degree, with college grads and even moreso postgraduate degree holders strongly favoring Democrats. OTOH, Blacks, who are overwhelming not elites, overwhelming are (and even more overwhelming lean) Democratic, and the same is true predominantly (though less overwhelmingly) of Hispanics. So, yes, to the urban part, not particularly independently true of the “elite” or “coastal” part.

wbeckler 6y ago

Here's another explanation, pre-election, of the gap, the found that Trump voters are more likely to lie to pollsters: https://www.cloudresearch.com/resources/blog/election-2020-poll-respondent-honesty/ https://www.cloudresearch.com/resources/blog/election-2020-p... These pollsters agreed with this phenomenon, arrived at from a totally different direction: https://www.politico.com/news/magazine/2020/10/29/2020-polls-trump-biden-prediction-accurate-2016-433619 https://www.politico.com/news/magazine/2020/10/29/2020-polls...

auganov 6y ago

As bad as election polls are, at least you get to find out the truth eventually. A lot of polling has to do with domains where there's no easy way of establishing ground truth. But ironically it would appear to me people tend to trust these polls more. While most seem to be skeptical about election polling, they might share exit polls on, say, demographics, without questioning their accuracy.

gridspy 6y ago

TL;DR: Distrustful people don't talk to strangers. They vote differently too.

bra-ket 6y ago
it's a bit hard to trust mainstream media that have been lying to you and called you 'racist' for 4 years.
- mring33621 6y ago
  Voting for racists makes you a racist.
  - bra-ket 6y ago
    define 'racist' and what makes you think Trump is a 'racist'
    - 0xffff2 6y ago
      "Shithole countries" and a refusing to denounce white supremacy make for an easy start.
      - bra-ket 6y ago
        He denounced white supremacy many times, you are misinformed. What ‘shithole countries’ have to do with racism
        colinmhayes 6y ago
        Denouncing white supremacists after the entire country says you have to isn't really the same. Anyone with critical thinking skills can see that the number of times he has refused to denounce white supremacists or called them good people makes him a white supremacist.

bra-ket 6y ago

>People who don’t answer polls .. tend to have low levels of trust in other people no, we just don't trust your propaganda machine

beervirus 6y ago

Here's a good illustration of just how one-sided the polling error was: https://twitter.com/thomasjwood/status/1325103908319064065 https://twitter.com/thomasjwood/status/1325103908319064065 The polls hugely and consistently underestimated the Republican votes.

jackcosgrove 6y ago
I would be interested to see these errors plotted against state population density.

johnnyg 6y ago

The book Thinking in Bets points out that predictions are given with a % of certainty. If I say X party will win at Y% and Z party wins, it doesn't mean the model is broken.

jerf 6y ago
On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!" Given two major elections in a row where the results were essentially out of the error bars, it is perfectly reasonable to be dropping your confidence on pollsters very hard. Once was perhaps understandable, but twice is getting into "right .01% of the time" territory.
- johnnyg 6y ago
  I hadn't considered the "unfalsifiable" aspect of my comment. I think you make a good point. At a poker table you get a high number of events to check against but a presidential election is only once every 4 years. I wonder how accurate the local elections are vs the national? If neither is accurate, then that's a strong case for your point of view and a change in methodology.
- cwkoss 6y ago
  > On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!" I wonder how many people still think Nate Silver has a credible perspective: he's spun the above strategy into a job in unscientific punditry.
rayiner 6y ago
But elections aren’t single events, they are 50 different events. 5% of the time the results in Michigan will be outside the error bars. But if the results are outside the error bars in Michigan, Iowa, Wisconsin, Minnesota, etc., and the polls all got it wrong in the same direction for each (overestimated Biden’s support) that’s actually statistically quite unlikely.

bryanlarsen 6y ago

Another pollsters opinion: Nate Cohn says the story is about Latino voters who are a notoriously hard demographic to poll. Source "The Daily" podcast.

SV_BubbleTime 6y ago
Did Nate Cohn's predictions this election pan out? “The final polls more or less comport with how we already viewed the race. Mr. Biden ends the race up by more than eight points nationwide — the largest lead a candidate has held in the final polls since Bill Clinton in 1996. He’s up by at least five points in states worth more than 270 electoral votes, the number needed to win. Beyond that, he’s got at least a nominal lead in states worth 350 electoral votes, and he’s just a 2012 polling error away from a sweeping landslide of more than 400 electoral votes.” https://www.nytimes.com/live/2020/presidential-polls-trump-biden/assessing-the-last-polls-before-the-election https://www.nytimes.com/live/2020/presidential-polls-trump-b... Cohn's polling leading up to the election: Tuesday Nevada Biden +6 Wednesday Michigan Biden +8 Thursday N.C. Biden +3 Sunday Arizona Biden +6 Sunday Florida Biden +3 Sunday Pennsylvania Biden +6 Sunday Wisconsin Biden +11 Which of these happened? Biden underperformed compared to Clinton in most pivot counties and bellweather predictions (which were 90% wrong this year, wow), and most counties in swing states entirely with the except of greaty overperforming in the densest liberal areas, in some of those cases even doing better than Obama as crazy as that seems. Biden underperformed with hispanics and black from exceptions and likewise Trump did better with them than expected overall. And no matter what if you think someone the day of the election saying "Biden Landside" was possible, well... You do you, but me personally, I'm going to go to someone else for opinions because that was such an exaggeration of reality that it seems like it had to be "for effect".
- bryanlarsen 6y ago
  Somebody who got it wrong and actually admitted he got it wrong is often a credible source on why he got it wrong.
  - SV_BubbleTime 6y ago

evo_9 6y ago

Who replies honestly to political question to random strangers? I know I don't. What's the benefit of letting anyone know where you stand politically at this point? None. It's much more likely - no matter which side you're for - that the person asking could have an adverse reaction. Things are just way too polarized now, not sure how that can change, but that's how it is. I mean outside friends/family I simply take a neutral, non-committal stance on almost every hot-button issue these days. And honestly even that's not 100% true, I avoid getting into deep political conversations with most of my family, I simply don't want to cause further rifts in our already frail relations over all this stuff. There is no middle ground.

dillondoyle 6y ago
Research has confirmed that people don't lie to pollsters in any meaningful way when asked their head to head preference. It's mentioned in the last graph of the article. The crux of this article is rather than 'shy trump voters' there is simply a large amount of these Trump voters who will refuse to take a survey flat out.
- colinmhayes 6y ago
  Isn't trump voters refusing to take polls what everyone means when they say "shy trump voters"?
  - PeterisP 6y ago
    As far as I understand, the traditional sense of "shy voter" is to describe your poll numbers (i.e. the people who did take the survey) which answered "undecided" when they were shy about their choice. This also goes hand in hand with things like observed differences in answers of in-person surveys when the person doing the survey is/isn't from the same demographic as the person being surveyed. A disproportional amount of people (after controlling for race/gender/income/education/etc) simply refusing to answer the survey has similar but conceptually different effect; one that (as the OP claims - perhaps they're mistaken) is new and had not been a significant factor before.

hashtagmarkup 6y ago

This isn't new information. Whoever is funding pollsters should know better, and pollsters themselves should be ashamed for taking the money. https://en.wikipedia.org/wiki/Bradley_effect https://en.wikipedia.org/wiki/Bradley_effect

rayiner 6y ago

> I reached out to the smartest polling guy I know: David Shor, an independent data analyst who’s a veteran of the Obama presidential campaigns who formerly operated a massive web-based survey at Civis Analytics before leaving earlier this year Before being fired for pointing out that violent protests alienate people: https://nymag.com/intelligencer/2020/06/case-for-liberalism-tom-cotton-new-york-times-james-bennet.html https://nymag.com/intelligencer/2020/06/case-for-liberalism-... Schor’s firing is a tremendous example of how free speech values make people better at their job, and attacks on free speech values can jeopardize organizations and political parties. It’s conventional wisdom now that failure to condemn violent protests earlier cost Democrats votes, including among minority groups. This was quite apparent from polling this summer. (By June, YouGov polls showed that Latinos were more worried about the “breakdown of law and order” than about “systemic racism.”) Creating a culture that makes people afraid to have dispassionate, context-appropriate discussions about what the data was saying made it harder for people to do their jobs effectively. Obviously what kinds of facts are relevant to an organization’s mission differs greatly depending on the organization! In many contexts Schor’s comments wouldn’t have been relevant and may have been better left unsaid to foster workplace harmony. But in a political organization, obviously some heated issues will come up and people need to be able to have honest discussions about them.

fastball 6y ago
"How did Trump get so many votes?" The same people that attacked Schor have undoubtedly also asked themselves this question, without an ounce of self-awareness. Except instead of recognizing their own culpability, the conclusion most such people are coming to seems to be "well, I guess nearly half of all Americans are bad people".
- flowerlad 6y ago
  Not bad people, but gullible people. Anyone that believes QAnon conspiracy theories such as Hillary Clinton, Bill Gates, and Oprah Winfrey are raping and drinking the blood of kidnapped children in pursuit of the “fountain of youth” is gullible.

ttamslam 6y ago

For no particular reason, I feel very strongly that you could achieve much more accurate results by asking people to estimate what _they_ think the final margin will be then average those results. I'm inclined to think that community members know their community well enough that given a large enough sample, this average will trend towards the true result. https://en.wikipedia.org/wiki/Wisdom_of_the_crowd https://en.wikipedia.org/wiki/Wisdom_of_the_crowd edit: A quick google search found: https://www.researchgate.net/publication/289826354_Predicting_elections_A_%27Wisdom_of_Crowds%27_approach https://www.researchgate.net/publication/289826354_Predictin...

mgraczyk 6y ago
You may find this specific technique interesting, in which you try to find the "expert minority" in the crowd, under the assumption that people who can better predict the crowd's belief can also better predict the true outcome: https://en.wikipedia.org/wiki/Surprisingly_popular https://en.wikipedia.org/wiki/Surprisingly_popular
- ttamslam 6y ago
  This is very interesting, thank you!
dillondoyle 6y ago
There's actually some good research on that. Another source - i think i've read a couple journal articles on this https://academic.oup.com/poq/article/78/S1/204/1836551#xref-corresp-1-1 https://academic.oup.com/poq/article/78/S1/204/1836551#xref-...
jedberg 6y ago
A good way to get a good prediction is to ask how they think their neighbor will vote. A person who is ashamed of their selection will be much more willing to tell you what it is if they think you're asking about their neighbors.

koiz 6y ago

People lie to pollsters or straight up won't admit who they voted for. Pollsters seem convinced otherwise.

AntiImperialist 6y ago

From election strategy point of view, you are in a much stronger position if you poll much higher, for various reasons: - uncaring and undecided voters, if they had to vote, are likely to vote for the winners, because being on the winning side makes them feel better (which may have decided this election) - if you lose the election, people are more likely to believe that the other side cheated if you were projected a clear winner by the polls, so you can bring people on the streets or run fraudulent lawsuits to deter the opponents and still get a lot of support So, strategically, it pays to poll higher regardless of how things really are. Given how a lot of media organizations are against Trump, it's not unlikely that these polling organizations were too... and they helped the opponents.

Animats 6y ago

Decades ago, phone polling got over a 50% response rate. Now it's under 10%. So there's at least a bias towards people who are willing to accept unsolicited calls.

istjohn 6y ago
The article says the response rate is just 1%.
- Animats 6y ago
  That must be newer data. At 1%, the data is mostly noise.

sien 6y ago

Tim Harford has an excellent article on why the polls were wrong: https://timharford.com/2020/11/why-the-polls-got-it-wrong/ https://timharford.com/2020/11/why-the-polls-got-it-wrong/ The big quote is this: "The pre-election numbers suggest that in a typical poll with 500 positive responses, 255 went for Mr Biden and 243 for Mr Trump. But typical response rates are 5 in 100 — often lower, says Andrew Gelman, a statistician and prominent election modeller. He says that only around 1 in 100 people respond to opinion polls. So now picture 10,000 people, 255 who called their vote for Mr Biden, 243 for Mr Trump and 9,500 who never responded. How confident are we feeling now? " Harford is the host of the excellent More or Less podcast that looks at the validity of statistics in the news.

drilldrive 6y ago

Frankly, this has an obvious answer: The people who answer polls are not representative of the population at large. Yet this was known decently enough since 2016. But more at the heart of the issue here is pollsters are providing a service not to inform but to persuade (both the public and donors); internal-polling is actually what matters. And instead of reformulating the service with more accurate questions for total populace (who will your neighbor vote for?), the pollsters doubled down on the standard methods and made (roughly doubly!) a larger polling error than 2016. I mean the Maine senate race was on average ~15 points off the final result[0]. Frankly this was the race for the pollsters to prove themselves that they can self-correct, as they have demonstrated to be unable to do this, it is clear they just should be abolished. [0] https://www.realclearpolitics.com/epolls/2020/senate/me/maine_senate_collins_vs_gideon-6928.html# https://www.realclearpolitics.com/epolls/2020/senate/me/main...

noncoml 6y ago
> it is clear they just should be abolished I am not sure I understand what you mean. Abolished how?
- drilldrive 6y ago
  What I mean to say is polling as a major industry for political campaigns is done after this election. The current people running these public polls have no place of confidence anymore: donors should not take them of value, and public opinion should ignore polling. They had their chance this time around for survival as an industry but roundly threw that away.

asdff 6y ago

A: sampling bias.

gshubert17 6y ago

I've kept a spreadsheet to compare the last polling estimates from fivethirtyeight.com with the actual election results. Although there may be 8 or 9 million votes left to count, the percentages are probably good enough to make some comparisons. What I saw was an average polling error of 4.9 percentage points. The actual results for Trump were 4.9 points higher than the polls. And the errors were greater in states that had higher percentages of votes for Trump. For example, North Dakota: The last polls showed 37.9% for Biden and 59.6% for Trump, a 21.7% difference. The latest popular vote tally from CNN shows 31.8% to 65.1%, which are 33.3% apart. That's an 11.6% error, much larger than expected sampling errors of 3-4%. Shor's explanation was interesting. Whether the explanation is right or not, I can see the effect Shor is describing.

mastazi 6y ago

This is my personal experience: over the course of my life, I have held both conservative positions and more progressive ones (depending on the topic, and in different periods of my life). I have always been much more willing to discuss openly those positions that I would describe as progressive. This is despite the fact that over the years I have been living in more than a handful of different countries, each one with different cultural norms. So I think there must be some sort of general tendency, where conservative opinions tend to be disclosed less frequently. I don't know why that is the case, and I would like to hear if anyone has an answer.

deleted 6y ago
[deleted]
fiblye 6y ago
I think it also depends on the audience and group you're with. If you're moving around a lot, you might end up in expat/international/diverse groups. Most people who move to several countries aren't moving to villages 4 hours from a city and speaking and blending in with the locals and really hearing what rural people think. Growing up in a small town and moving between small towns across the US every year or so, I never had exposure to "liberals" until I started using the internet as a teen. Until then, everyone I was ever around talked about them like they were bogeymen. In the country I live in now, I was initially exposed to much more "progressive" opinions from people. Now I'm seen as less "that foreign guy" and more of a local, and I hear what people would normally discuss with their friends and family. It's quite different.
- mastazi 6y ago
  It's true that context counts, I can see your point since, similar to your experience, I also come from a rural area and I lived in a tiny town for some years. But I also feel like I have a "default setting" for situations where I don't really know who I'm talking to, and in those situations, what I said above still applies.
apta 6y ago
> I don't know why that is the case, and I would like to hear if anyone has an answer. Cancel culture and SJW mobs.

jungletime 6y ago

Dr.SHIVA Ayyadurai, MIT PhD. "Inventor of Email" Analyzed the results in Michigan and came to the conclusion that 69k votes were taken from Trump and given to Biden. His reasoning here: https://www.youtube.com/watch?v=Ztu5Y5obWPk&feature=youtu.be https://www.youtube.com/watch?v=Ztu5Y5obWPk&feature=youtu.be

ZephyrBlu 6y ago
Do you have a TL;DW? I would watch it all if it was a bit shorter or there were timestamps, but 1hr10min is a lot.
- jungletime 6y ago
  From my understanding: He found that the more a republican a county is, the more republicans voted for Gop candidates, and against Trump. But only after the first 20% or so of the vote was in. In the first 20% of the vote Trump was doing better than the Gop candidates on the ticket. Comes to the conclusion that an algorithm is syphoning votes from Trump to Biden. In a predictable linear fashion after the first 20% of the vote is counted. And flips more votes in more republican counties in an almost perfect linear way. It doesn't make sense because Trump was polling better than the Gop candidates before the election, and was popular with republican voters. The tallying software has a weighted vote feature. Where votes can be weighted for example 2:1 for one candidate over the other. Thinks that this was kicked in after 20% of the vote was in. Which would produce a linear drop that the graphs show.
rjkennedy98 6y ago
That was not convincing at all and had some really glaring errors. Dr Shiva somehow confuses percent of overall voters with percent of Republicans to make it seem like there is a conspiracy going on. He says that margin for Trump should be a flat line so that if there 5% republican all-ballot voters it should be off by the same margin in Trump vote as a precinct with 80%. However if 20% of republicans flip that would mean 1% flipped vs 16% in those examples which is a line with a slope. Literally flipping those graphs would make them look the same for Biden as for Trump. All it shows is basically a kind of regression to the mean. Precincts with high republican or democratic all-ballot voting weren’t as republican or democratic as they seem. There could be many good reasons for that such as that voters that go against the way their communities vote tend to not be all-ballot voters. Also, Dr. Shiva is not an independent voice. He ran in Massachusetts as a Republican.

DecoPerson 6y ago

“The kind of people who answer polls are really weird, and it’s ruining polling.” This is true everywhere. I’ve been involved polling in for a popular AAA game, a small retail business, school events, online forums, and more; they all fail to account for the “who chooses to answer” bias (is there a name for this?). I speculate that the “average 4.7 million viewers“ rating Firefly received when it aired was totally off. The Fox executives (who clearly didn’t care for the show, if you read the full story) used this as ammunition to have it cancelled. (see https://en.m.wikipedia.org/wiki/Nielsen_ratings https://en.m.wikipedia.org/wiki/Nielsen_ratings for criticisms of the rating system used at the time). The people who would take the time to answer a poll (or connect a device to their TV for some cash) are a small subset of people who will skew your data in unintuitive ways. If more business people understood this, our world would be a better place.

euank 6y ago
> is there a name for this bias? I think it's response bias. https://en.wikipedia.org/wiki/Response_bias https://en.wikipedia.org/wiki/Response_bias
- devinplatt 6y ago
  I think actually op is referring to https://en.wikipedia.org/wiki/Participation_bias https://en.wikipedia.org/wiki/Participation_bias: "Participation bias or non-response bias is a phenomenon in which the results of elections, studies, polls, etc. become non-representative because the participants disproportionately possess certain traits which affect the outcome." Confusing terminology, but I guess "response bias" is about a false response by the participant to a question, and "non-response bias" is about whether the selected participant responded (participated) at all.

saeranv 6y ago

To me, this article highlights how much of a moving target weighting polls, and political prediction is. Shor here speaks about how Democrats get oversampled as both in 2016, and 2020 due to different levels of engagement and trust over the election: >> It used to be that once you control for age and race and gender and education, that people who trusted their neighbors basically voted the same as people who didn’t trust their neighbors. But then, starting in 2016, suddenly that shifted. If you look at white people without college education, high-trust non-college whites tended toward [Democrats], and low-trust non-college whites heavily turned against us. In 2016, we were polling this high-trust electorate, so we overestimated Clinton. These low-trust people still vote, even if they’re not answering these phone surveys. Is this shift partially because Trump and then QAnon have shifted Trump-voters trust in institutions - thus the voters that don't respond to pollsters are shifting as the Republican party becomes more radicalized?

RickJWagner 6y ago

My theory is that it's the High School Effect. Democrats have highly effective social messaging, led by Hollywood and professional athletes. They constantly put out messaging, making dissenters appear 'uncool'. Sad to say, adults are actually swayed by such childish forces. How else to explain the constant error to the left in polls?

ab3rC1te 6y ago

It's called cheating, fake news, and whatever else you want to deem it as. That's why the Postal service got a 10 billion stimulus package. Just like COVID is so dangerous that it jumps 26 ft in cold air, punches your plexiglass cause it wants to attack you, and put that mask on, it can't go through that either. Even though it's air borne and been air borne since it was released. Just look up and see what's being sprayed in the atmosphere to rain down COVID 21 on you. When will people stop being idiots to the bullshit lies that they feed you.

EGreg 6y ago

This is ridiculous! Here I am telling people that Republicans are distrustful of mainstream media, science, colleges etc. and that pollsters must know what they're doing, getting representative samples is the best we can do etc. And then I found out what I suspected is true. The most trivial thing of some people being overrepresented in a poll by the mere fact that they agree to take one, apparently, was somehow TOO MUCH FOR THE POLLSTERS TO HANDLE. Are you kidding me? The methodology should involve getting truly REPRESENTATIVE and RANDOM samples. While each one is going to be biased in some way, the bias should be independent of political affiliation. For example, stopping people on a street. Yes it excludes people who don't go out on the street, or people who won't stop. Then you are supposed to note how many people refused to take the poll. And then you are supposed to measure the correlation between that and their beliefs. If you really can't do this, then have a sign outside the voting areas, as people exit, with Donald Trump or Biden or JoJo etc. and have them sign up for polling in the next four years. This way at least you can make sure going in that your population is balanced as of four years ago.

mmcgaha 6y ago

Everyone knows Republicans are racist homophobic xenophobic bigoted nazis. Why would they want to admit that to some random over the phone? The rhetoric that we are using to describe the other party is not necessarily helping our cause in the long run. First it is pushing folks with legitimate disagreement away from public discourse. Second it is creating a good-guy/bad-guy scenario that is the equivalent of tribalism. At this point I can pick between a political talk show or professional wrestling and get about the same quality of content.

bArray 6y ago
You should really edit your comment's first paragraph to make it clear that it's not your opinion. I agree that there is currently an atmosphere where people don't necessarily feel comfortable talking about their political leanings among strangers.
- bananadrama 6y ago
  > You should really edit your comment's first paragraph to make it clear that it's not your opinion. It's clear from context.
  - nikitaga 6y ago
    My eyes roll faster than context arrives. Deliberately appearing as a troll until the crucial context arrives to reverse the entire meaning is a shit narrative device when discussing serious issues. Save this stuff for stupid memes.
    - JeremyNT 6y ago
      > Deliberately appearing as a troll until the crucial context arrives to reverse the entire meaning is a shit narrative device when discussing serious issues. Save this stuff for stupid memes. Even with the context it's obviously a troll, because the poster spends the entire first graph building a straw man. The underlying implication is that Republicans are somehow persecuted against by a nameless group who believes all of these negative attributes apply, but no evidence is presented for this.

martyvis 6y ago

"The voter approaches their table, and gives their name and party affiliation. The party affiliation is used as a signal that something fishy might be going on if the ratio of R to D is very different from the votes recorded." This sounds so weird to an Aussie. For most people when you enter the polling place, you get bombarded by party supporters with their "how to vote" leaflets. Most people take all or none to be polite and not indicate your preference. Inside you give your name and it is ruled through on the roll printout and you are handed your ballot papers. Once you vote in your little cardboard both you slot the paper in the box. You go outside, hand back the how to vote leaflets, and chat with people after you have bought your "democracy sausage" ( a hot dog bought from the local school or fire service fundraiser). No one really cares how you voted (and 95% of people eligible vote)

AtlasBarfed 6y ago
Do you guys have voting days off?
- izacus 6y ago
  I'm guessing the vote is on Sundays?

AtlasBarfed 6y ago

If pollsters from a long and academic history are getting this wrong, and it basically is a two-sided yes/no question, what validity is there in twitter/facebook mining sentiment analysis?

tchock23 6y ago

One potential solution is to pay people to participate in polls. Pollsters don’t do this because it is prohibitively expensive and cuts into their consulting margins, but it can reduce sampling bias (it can also introduce a bias if not used effectively).

Kylekramer 6y ago
Online pollsters mostly "pay" their respondents, and did just as poorly as phone pollsters this go around.

disown 6y ago

I wonder what Facebook or Google's "poll" numbers would have been if they decided to run one? Given their access to search, texts, social media posts, emails, tracking, etc along with their profile of a large segment of the population, I'm guessing their polling data would be the most accurate.

axaxs 6y ago
Facebook already determines your political leaning, somehow. At least a few years ago you could see it for yourself, not sure if that still exists. I'm sure without a shadow of a doubt Google makes the same determination. So they probably don't need to 'run' anything special here.

burlesona 6y ago

> Qualitative research doesn’t solve the problem of one group of people being really, really excited to share their opinions, while another group isn’t. As long as that bias exists, it’ll percolate down to whatever you do. Pretty good TLDR from the end of the article.

diebeforei485 6y ago

Why were the errors only in one direction?

_fizz_buzz_ 6y ago
Because they are systematic errors e.g. Latinos are hard to poll. Latino vote went more in the direction of Trump then previously. So, every pollster is running into that issue.

ospohngellert 6y ago

Does anyone else reject the premise of the article? I understand that polling definitely missed big in some areas, but isn't that normal? Polling is really hard. You have to make a lot of assumptions, adjust for demographics, predict whether a voter will turn out, while using a sample of usually ~1000 people. Of course some polls are going to be wrong. I don't think the polling was generally wrong though. Polling averages had Biden +8 or so, when all votes are counted he'll likely be +5, which is within margin of error. Polls were extremely accurate in Georgia, where conventional wisdom would say Biden shouldn't win. I get that most of the polling error benefited Trump, and the industry needs to reflect on why that happened and how to prevent underestimating republicans again in the future, but I don't yet buy the narratives that "the polls got it wrong."

deleted 6y ago

[deleted]

chmod600 6y ago

Is there any solution? Perhaps using more proxies and indicators rather than direct questions?

scoot_718 6y ago

or voter suppression, which polling doesn't take into account.