7 ms·

One of the biggest issues I think is that the polling is still done by just randomly calling people. I think I agree that there is a large skew introduced by th

• deleted 6y ago
  [deleted]
• legitster 6y ago
  This isn't strictly true though, there are any number of polling methods that pollsters use these days including online polling and essentially pre-approved call databases. One of the examples in this article was an in-person poll with 70% response rate. It's probably more accurate to say it's harder to effectively weigh or average responses in a setting where there are 4 or 5 different channels you are collecting responses from.
  • samth 6y ago
    The GSS spends (according to Shor) $1000 per response. It's not at all similar to the "polls that were wrong" in the election.
  • pmiller2 6y ago
    That’s interesting, considering I would be far more likely to dodge an in-person poll than any other type. If I see anyone with a clipboard asking people if they’re registered voters, I give a wide berth.
  • flomo 6y ago
    Public radio once had a guest describing their online polling methodology - it was running "Win a Prize" type banner ads. So probably not very representative either.
• koolba 6y ago
  Even the method of contact would skew toward people who have a means for the method of contact or location. Cell phones might include less seniors. Questioning people walking into Whole Foods is going to give you a different set than people walking into Cracker Barrel. If the source set of names and numbers is truly representative, I think random calling could work if you make sure to include "did not response" as a line item. Then you might get something like: > 45% Biden, 39% Trump, 1% Jorgensen, %15 ??? With the no-response removed you get a skewed: > 52% Biden, 45% Trump, 1.1% Jorgensen
  • jerf 6y ago
    I suspect "did not respond" is already more than 15% and increasing constantly. I could easily see it already being 50%+. Even a layman would know to be suspicious of "12% Biden, 11% Trump, 77% No Response". I can't even tell you if a pollster called me... I'm getting about half-a-dozen calls a day from rando numbers that want to refinance my nonexistent student loans or talk to me about my home's nonexistent warrantee. I went to permanent do-not-disturb months before the election was even in the news. If you don't leave me a voice message I don't even know you exist; if you do leave me a voice message you've got about 3 seconds to convince me you're not a spammer before it's trashed.
    • rayiner 6y ago
      The “no response” is actually more like 80-90%. See: https://www.nationalreview.com/podcasts/the-editors/special-episode-the-pollster-who-thinks-trump-is-ahead/ https://www.nationalreview.com/podcasts/the-editors/special-...
• ianleeclark 6y ago
  The thing that not enough people bring up is the rampant robocalling. Before I left the US around a year and a half ago, I was getting 2-3 calls a day. Basically, if I ever got a call from a number not in my contact list, I would never answer. I know I'm not the only one like this, but I don't have any numbers to back it up. Even in Germany where robocalling seemingly doesn't exist, I still have this reflex, so it makes me seriously consider the validity of any phone-based polling.
  • losteric 6y ago
    > Basically, if I ever got a call from a number not in my contact list, I would never answer. I know I'm not the only one like this, but I don't have any numbers to back it up. huh... I have literally never met a single person that answers cold calls on personal lines, this was already SOP when I was a kid. Seriously, who's picking up those calls?
    • throwaway894345 6y ago
      This wasn't SOP for me when I was growing up. We never got spam calls. Only the occasional wrong number.
      • pmiller2 6y ago
        +1 to this, although I can say that if I had caller ID, I probably wouldn’t have answered unknown numbers, even when I was growing up.
    • andybak 6y ago
      > Seriously, who's picking up those calls? People who don't live in countries where the problem is this bad? I'm in the UK and I get about 1 unsolicited call a month. If I didn't pick up numbers I didn't recognize I would have missed my Doctor (this week), several new client leads and multiple friends and acquaintences that changed their number.
      • throwaway201103 6y ago
• tomjen3 6y ago
  That would seem likely. But then the polls should undercount a population that is more likely to vote for the Dems and so the polls should have shown more support for the republicans than what happened in the elections. Maybe we should just have taken the lesson of Dewey defeating Truman and not trusted polls is the first place.
  • esoterica 6y ago
    The polls were not great this cycle, but the 538 forecast, which is mostly polls-based, still correctly predicted 48/50 of the states in this election. If you went an entire cycle without looking at a single poll, or reading a single piece of media informed by a poll, do you think you could do better than 48/50?
    • icedistilled 6y ago
      Correctly predicting is not a badge of honor if the numbers are way off. As an extreme hypothetical, if he'd predicted California to be Biden+40 and it was actually Biden+10 that would be correctly predicting who won but it would also be a massive polling failure. If the numbers land outside the predicted margins, it's bad polling regardless of whether the binary decision was right.
• dragonwriter 6y ago
  > One of the biggest issues I think is that the polling is still done by just randomly calling people It's done by a number of means, including online. IIRC, live caller polls still tend to produce the best results, though.
• mring33621 6y ago
  Agreed. I would also suspect that there are some other factors: - people hoping for change are more willing to answer polls - people who are ashamed of their opinions are more unwilling to answer polls The second one, perhaps, gives too much credit for self-awareness, though.
  • bra-ket 6y ago
    I would also suspect that there are some other factors: - people who have too much free time on their hands will answer polls more - gullible people, who tend to click on spam emails and ads and trust their news, will answer polls more - people who've been vilified by the media for their views are unlikely to answer polls - people who are afraid to be 'cancelled' or harassed for their views are unlikely to answer polls - people who don't trust particular news organizations spreading political propaganda are unlikely to answer polls from those organizations
  • goostavos 6y ago
    Aaaaand you've actually answered your own question. Why would people not give an actual opinion to a stranger when there's this default opinion of them as (a) having wrongthink they should be ashamed of, or (b) too stupid to realize their wrongthing is in fact wrong? Guess we'll never know why those dummies would possibly hide their true preferences.
  • dillondoyle 6y ago
    It's less about 'ashamed of their opinions' or the 'shy trump voter.' The article talks about this. The crux of the problem - at least argued by this authored and from what I see working politics backed up generally - is that those Republican / Trump voters do not trust others and are less likely to take the poll in the first place. Those that do take survey generally do not lie about their support. The article gives evidence. Here is the section: "I use the General Social Survey’s question, which is, “Generally speaking, would you say that most people can be trusted or that you can’t be too careful in dealing with people?” .... It turns out, in the GSS, that 70 percent of people say that people can’t be trusted. And if you do phone surveys, and you weight, you will get that 50 percent of people say that people can be trusted. It’s a pretty massive gap. [Sociologist] Robert Putnam actually did some research on this, but people who don’t trust people and don’t trust institutions are way less likely to answer phone surveys. Unsurprising! This has always been true. It just used to not matter."
• bootlooped 6y ago
  Most/many polls weight for this. If the people they reach are 60% seniors and 40% non-seniors, they know that seniors don't make up 60% of the population, so they weight based on likely voter demographic numbers. After 2016 they also started weighting on education level, because they thought that was a factor in the polling miss of 2016.
  • redisman 6y ago
    Weighing of course can add much more uncertainty. What if there is a portion of the country you simply can't reach for some reason, or you hear back from just 2 people? That data would be as good as useless even if you extrapolate with weights.
• Dumblydorr 6y ago
  Exactly this. How many Cuban americans in Florida had phones that they answered? The polls clearly missed this demographic group and it's pure "malarkey" to say they were shy Trump voters. The amount of times I've tried to debunk this shy voter nonsense...its much more likely that random calls miss people than they are shy to admit their support anonymously.
• ThePadawan 6y ago
  I recently learned that the German polls also rely on random calls - but worse than that, they only use the publicly registered phone numbers, i.e. the ones in the phone book. That list mostly includes landlines, and biases towards older landlines as well (since IIRC phone book registration switched from opt-out to opt-in at some point). So the polls strongly overemphasize older voters (because no one else has a landline phone anymore).