5 ms·

To put it in perspective, the disparity is equivalent to polling 10K americans and extrapolating to all of america (which, for better or for worse, is what most

To put it in perspective, the disparity is equivalent to polling 10K americans and extrapolating to all of america (which, for better or for worse, is what most pollsters do).

ProCynic 13y ago
If the sample is truly random, it doesn't matter what the population size is. Of course, truly random samples are hard to get.
- adw 13y ago
  That depends on the population variance.
- deleted 13y ago
  [deleted]
akavi 13y ago
Except pollsters put in a lot of effort in making sure that the sample is representative of the larger population. Which is not true here, as the GP correctly notes.
- mikeash 13y ago
  Do they really? My only interaction with pollsters has been either having them call me or solicit me on the street, and both approaches have a tremendous inherent bias. As far as I know, this is how the big national agencies do things.
  - PeterisP 13y ago
    There are all kinds of adjustments done afterwards to correct for various factors. For example, you know the age distributions in USA; and if you find out your phone calls are getting twice as many seniors than the proportion should be[1], then you throw away a random parts of them so that they don't skew final 'data' towards the typical opinions of seniors. [1] Assuming that you're not measuring average age or measuring 'who is at home', but if you want to see, say, the average political opinion of total USA population, which tends to correlate with age.
    - mikeash 13y ago
      It still seems impossible to correct for everything. Sure, you could correct for age as you describe, but I imagine that landline phone ownership correlates with political opinions in all sorts of other ways too. Furthermore, how do you gather the data needed to correct the polling numbers without being able to accurately poll people in the first place? Seems like a complete chicken-and-egg problem.
      - PeterisP 13y ago
        "how do you gather the data needed to correct the polling numbers" -> you use the census. You need some info about the total population, you get it periodically and it doesn't change that much; you don't need to repeat it for every survey.
        mikeash 13y ago
        The census doesn't tell you about most of the godzillion factors that link landline phone ownership with political opinions.
Zikes 13y ago
I often see polls of 1K Americans and extrapolating out from there. An example: http://www.webpronews.com/americans-think-cloud-computing-comes-from-actual-clouds-2012-08 http://www.webpronews.com/americans-think-cloud-computing-co... Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million.
- gnaritas 13y ago
  > Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million. If it's a proper random sample, then it's far better than sampling all 319 million because it's 95% accurate with about a 3% margin of error and vastly cheaper and actually practical; you can't poll 319 million people.
mikeash 13y ago
It's counterintuitive, but the sample size needed for a good measurement doesn't much depend on the size of the overall population. What matters is getting a properly random sample. This is where pollsters fall down, because their "random" sample tends to be heavily biased toward the sort of person who has a landline telephone and doesn't hang up on pollsters. Polling 10,000 Americans would be vast overkill, in any case.
deleted 13y ago
[deleted]
gnaritas 13y ago
You say that as if 10k is too small a sample size to extrapolate accurately with, but actually that's a huge sample size that if done properly would be extremely accurate. You don't need to poll anywhere near 10k people to accurately predict all Americans views. You can sample less than 2000 people and get 99% accuracy with a 3% margin of error for a population of 325 million. Increasing the sample size to 10k simply reduces the margin of error to 1.29%, hardly worth the extra sampling of 8k people.
auctiontheory 13y ago
If you haven't read Asimov's Election Day, I highly recommend it.

To put it in perspective, the disparity is equivalent to polling 10K americans and extrapolating to all of america (which, for better or for worse, is what most pollsters do).

ProCynic 13y ago

If the sample is truly random, it doesn't matter what the population size is. Of course, truly random samples are hard to get.

adw 13y ago
That depends on the population variance.
deleted 13y ago
[deleted]

akavi 13y ago

Except pollsters put in a lot of effort in making sure that the sample is representative of the larger population. Which is not true here, as the GP correctly notes.

mikeash 13y ago
Do they really? My only interaction with pollsters has been either having them call me or solicit me on the street, and both approaches have a tremendous inherent bias. As far as I know, this is how the big national agencies do things.
- PeterisP 13y ago
  There are all kinds of adjustments done afterwards to correct for various factors. For example, you know the age distributions in USA; and if you find out your phone calls are getting twice as many seniors than the proportion should be[1], then you throw away a random parts of them so that they don't skew final 'data' towards the typical opinions of seniors. [1] Assuming that you're not measuring average age or measuring 'who is at home', but if you want to see, say, the average political opinion of total USA population, which tends to correlate with age.
  - mikeash 13y ago
    It still seems impossible to correct for everything. Sure, you could correct for age as you describe, but I imagine that landline phone ownership correlates with political opinions in all sorts of other ways too. Furthermore, how do you gather the data needed to correct the polling numbers without being able to accurately poll people in the first place? Seems like a complete chicken-and-egg problem.
    - PeterisP 13y ago
      "how do you gather the data needed to correct the polling numbers" -> you use the census. You need some info about the total population, you get it periodically and it doesn't change that much; you don't need to repeat it for every survey.
      - mikeash 13y ago
        The census doesn't tell you about most of the godzillion factors that link landline phone ownership with political opinions.

Zikes 13y ago

I often see polls of 1K Americans and extrapolating out from there. An example: http://www.webpronews.com/americans-think-cloud-computing-comes-from-actual-clouds-2012-08 http://www.webpronews.com/americans-think-cloud-computing-co... Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million.

gnaritas 13y ago
> Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million. If it's a proper random sample, then it's far better than sampling all 319 million because it's 95% accurate with about a 3% margin of error and vastly cheaper and actually practical; you can't poll 319 million people.

mikeash 13y ago

It's counterintuitive, but the sample size needed for a good measurement doesn't much depend on the size of the overall population. What matters is getting a properly random sample. This is where pollsters fall down, because their "random" sample tends to be heavily biased toward the sort of person who has a landline telephone and doesn't hang up on pollsters. Polling 10,000 Americans would be vast overkill, in any case.

deleted 13y ago

[deleted]

gnaritas 13y ago

You say that as if 10k is too small a sample size to extrapolate accurately with, but actually that's a huge sample size that if done properly would be extremely accurate. You don't need to poll anywhere near 10k people to accurately predict all Americans views. You can sample less than 2000 people and get 99% accuracy with a 3% margin of error for a population of 325 million. Increasing the sample size to 10k simply reduces the margin of error to 1.29%, hardly worth the extra sampling of 8k people.

auctiontheory 13y ago

If you haven't read Asimov's Election Day, I highly recommend it.