4 ms·
There's scope for pretty massive sampling errors with any door to door survey like this Not necessarily. While it's common for people on HN to eschew the resu
by reaperducer 7y ago
There's scope for pretty massive sampling errors with any door to door survey like this
Not necessarily.
While it's common for people on HN to eschew the results of surveys because of presumed sample bias, that's rarely what happens.
I took a couple of statistics classes in college that spelled out the math involved, and essentially if you survey enough people within a population of a certain size, unintentional bias gets reduced to the point where it becomes a non-factor.
It's all very interesting if you're into math. I am not, but statistics courses were required for my journalism degree.
("Unintentional" meaning there is such a thing as "intentional" bias. For example, if you're deliberately surveying Toyota owners about Toyotas, or something like that.)
- cookie_monsta 7y agoYes, I don't know the maths but this study surveyed 2,000 people from a pool of around 54 million (18% of Britain's 66 million people are under 16 [1]). I don't think those numbers qualify as being enough people within a population of a certain size to avoid unintentional bias. [1] https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/articles/overviewoftheukpopulation/november2018 https://www.ons.gov.uk/peoplepopulationandcommunity/populati...
- reaperducer 7y ago2,000 might actually be oversampling for 66,000,000 people. When the United States had about 300,000,000 people the number for statistical veracity was around 975. Again, my statistics classes were a long time ago, so the numbers I remember are inexact. But once I groked the math, it was surprising how few people you needed to interview.
- cookie_monsta 7y agoYes, there is a comment further down about how population size doesn't matter, that all you need is a sample size of around 1,000. Fascinating. I stand corrected.
- 0xffff2 7y agoI too have taken a couple of college statistics classes. I think you're misunderstanding/misrepresenting things a bit here. Yes, the internet at large loves to dismiss survey results for any and all perceived flaw, no matter how minute. That said, they aren't always wrong. In this particular case, everything you've said is true only if you assume a reasonably random sample. If your sampling method itself introduces biases, no increase in sample size is going to eliminate those biases. I'm not enough of a statistician to feel particularly confident in passing judgment over the sampling method here, but I think it's fair to question it, despite the fact that I'm ordinarily far more likely to take the statistics at face value than the average internet commenter.