9 ms·

> > But if a sample is random and representative, you could estimate population statistics of millions (or even infinite) with a sample of just 30 or so. > Whi

> > But if a sample is random and representative, you could estimate population statistics of millions (or even infinite) with a sample of just 30 or so. > While that statement is technically correct, it is useless in practice. I don't really follow your argument for small sample sizes being useless in practice. Isn't the whole point of things like confidence/credible intervals, the bootstrap, and other common statistical methods, to quantify the uncertainty of estimates due to the sampling procedure, including sample size and the possibility of outliers? And, while we may not know that our sample is representative, isn't random sampling from the population of interest a method that will generate a sample that is representative with high probability? That these methods are well known in statistical practice suggests that samples are generally useful, even when they are not large. The existence of pathological distributions where we might get misleading results (e.g., the classical "wealth over the people in a room if you throw Bill Gates in there") doesn't make these methods useless, does it?

p-e-w 3y ago
> isn't random sampling from the population of interest a method that will generate a sample that is representative with high probability? That depends on the sample size. The comment I replied to claimed that "you could estimate population statistics of millions (or even infinite) with a sample of just 30 or so". Which is certainly not true if the sample is drawn randomly, except under very specific circumstances (such as population variance being extremely small). You don't need a billionaire in order for income distribution samples to be wildly misleading. It's enough for the variance to be high, so that very small samples are almost guaranteed to have a mean that differs substantially from that of the whole population.
- tmoertel 3y ago
  Draw at random just six people from a population of millions. Take the min and max wealth of those six people. That procedure produces a confidence interval that contains the true median wealth of the population more than 95% of the time. Even if the population wealth variance is high. How did I just produce a reliable estimate using a sample size of only six? By quantifying the uncertainty of the estimate. There's a trade off between sample size and uncertainty, and that trade off allows small sample sizes to be useful for many practical applications, most of which can accommodate uncertainty. Again, small samples are informative. Whether they are informative enough for a particular application doesn't determine whether they are informative enough for all applications in general. I certainly wouldn't call them "useless in practice."

p-e-w 3y ago

> isn't random sampling from the population of interest a method that will generate a sample that is representative with high probability? That depends on the sample size. The comment I replied to claimed that "you could estimate population statistics of millions (or even infinite) with a sample of just 30 or so". Which is certainly not true if the sample is drawn randomly, except under very specific circumstances (such as population variance being extremely small). You don't need a billionaire in order for income distribution samples to be wildly misleading. It's enough for the variance to be high, so that very small samples are almost guaranteed to have a mean that differs substantially from that of the whole population.

tmoertel 3y ago
Draw at random just six people from a population of millions. Take the min and max wealth of those six people. That procedure produces a confidence interval that contains the true median wealth of the population more than 95% of the time. Even if the population wealth variance is high. How did I just produce a reliable estimate using a sample size of only six? By quantifying the uncertainty of the estimate. There's a trade off between sample size and uncertainty, and that trade off allows small sample sizes to be useful for many practical applications, most of which can accommodate uncertainty. Again, small samples are informative. Whether they are informative enough for a particular application doesn't determine whether they are informative enough for all applications in general. I certainly wouldn't call them "useless in practice."