5 ms·

They say "That being said, the sample, while not scientifically random, is double what’s considered statistically relevant for the total population of Java deve

They say "That being said, the sample, while not scientifically random, is double what’s considered statistically relevant for the total population of Java developers worldwide." Great. So to forecast the next presidential election they're going to poll a million NRA members and celebrate the fact that the sample size is much greater than what's needed.

mitchty 13y ago
Yeah, if its not actually random its not going to be all that useful a sample size.
skj 13y ago
Right, apparently the notion of sample bias escaped whomever said that.
- grkvlt 13y ago
  No, they are stating that they technically have a large enough sample for the results to be meaningful if the sample were unbiased (what they call 'scientifically random') - a large enough sample size is a necessary but not sufficient condition to allow valid statistical inferences to be drawn.
pessimizer 13y ago
What's 'scientifically' random? Is that like normal random, but with its highfallutin' nose stuck up in the air?
- rtkwe 13y ago
  Scientifically random would be choosing data points such that each member of the target population, Java developers, has an equal chance of being surveyed. This blog's survey has a couple biases, one is self selection (it was only posted on the blog so only people who read the blog have responded).
- tel 13y ago
  When drawing a survey population that you want to use to extend knowledge to the total population you need to sample in a way that doesn't introduce bias. The Simple Random Sampling method is the gold standard here: sample in a way such that each subset of the total population of a given size has equal chance of being selected. This includes, as a special case, that each individual has an equal chance of being selected. http://en.wikipedia.org/wiki/Simple_random_sample http://en.wikipedia.org/wiki/Simple_random_sample More complex sampling methods can be used since SRS is usually intractable (you'd need to put every single person in your superpopulation into a lottery by magic, else you'll bias toward those who like email surveys or actually read their mail). In the event you use one of these you usually need a bias mitigation strategy. For instance, you might just state something like "we assume that receptiveness to email and dislike for Java are correlated". It's then up to the reader to decide whether they trust your assumptions. You might also do some other support studies which help to measure the size of the bias your non-SRS sampling methodology induces and then correct for them.
dragonwriter 13y ago
In addition to the problem you point to (size of a sample is irrelevant if it isn't random), statistical relevance of a sample is based on the size of the sample alone, not the size of the population, so their claim that it is "double what's considered statistically relevant for the total population of Java developers worldwide" is nonsense.

mitchty 13y ago

Yeah, if its not actually random its not going to be all that useful a sample size.

skj 13y ago

Right, apparently the notion of sample bias escaped whomever said that.

grkvlt 13y ago
No, they are stating that they technically have a large enough sample for the results to be meaningful if the sample were unbiased (what they call 'scientifically random') - a large enough sample size is a necessary but not sufficient condition to allow valid statistical inferences to be drawn.

pessimizer 13y ago

What's 'scientifically' random? Is that like normal random, but with its highfallutin' nose stuck up in the air?

rtkwe 13y ago
Scientifically random would be choosing data points such that each member of the target population, Java developers, has an equal chance of being surveyed. This blog's survey has a couple biases, one is self selection (it was only posted on the blog so only people who read the blog have responded).
tel 13y ago
When drawing a survey population that you want to use to extend knowledge to the total population you need to sample in a way that doesn't introduce bias. The Simple Random Sampling method is the gold standard here: sample in a way such that each subset of the total population of a given size has equal chance of being selected. This includes, as a special case, that each individual has an equal chance of being selected. http://en.wikipedia.org/wiki/Simple_random_sample http://en.wikipedia.org/wiki/Simple_random_sample More complex sampling methods can be used since SRS is usually intractable (you'd need to put every single person in your superpopulation into a lottery by magic, else you'll bias toward those who like email surveys or actually read their mail). In the event you use one of these you usually need a bias mitigation strategy. For instance, you might just state something like "we assume that receptiveness to email and dislike for Java are correlated". It's then up to the reader to decide whether they trust your assumptions. You might also do some other support studies which help to measure the size of the bias your non-SRS sampling methodology induces and then correct for them.

dragonwriter 13y ago

In addition to the problem you point to (size of a sample is irrelevant if it isn't random), statistical relevance of a sample is based on the size of the sample alone, not the size of the population, so their claim that it is "double what's considered statistically relevant for the total population of Java developers worldwide" is nonsense.