3 ms·

> their sample size (6,000) is pretty small to be generalizing to all Americans Not really. It has been a while since I took a statistics class, but a sample s

by objclxt 10y ago

> their sample size (6,000) is pretty small to be generalizing to all Americans Not really. It has been a while since I took a statistics class, but a sample size of around 4,00 will give you a 99% confidence rating for the entire US population with a 2% margin of error [1]. Of course, you need to pick the participants correctly, but the size alone is more than enough to generalize to the entire population. I would be more suspicious of the fact the survey was run by the Freelancers Union and Upwork, two organizations who directly benefit from more freelancers. So it's somewhat in their best interests to get a high result. [1]: https://en.wikipedia.org/wiki/Sample_size_determination https://en.wikipedia.org/wiki/Sample_size_determination and https://www.surveymonkey.com/mp/sample-size-calculator/ https://www.surveymonkey.com/mp/sample-size-calculator/

mdorazio 10y ago
I'm aware of sample size determination, but minimum sample sizes typically assume random sampling over the target population. All their sampling methodology slide says is that it was an online survey of adults who have done some paid work in the last 12 months, and they did some weighting to match BLS stats. We have no idea how they found the participants or controlled for bias in the selection process.
- markcampbell 10y ago
  This is a very important aspect of the study. They did not do a true randomized survey as participants were self-selecting (online surveys have this inherently -- well, mostly).
  - dredmorbius 10y ago
    It's not clear that this was or wasn't a self-selecting sample. The article does state that the surveying was conducted online: In the research, more than 6,000 U.S. working adults over the age of 18 were surveyed online by independent research firm Edelman Intelligence between July 29 — August 24, 2016. Among them, 2,049 were freelancers and 3,953 were non-freelancers. The survey defined freelancers as individuals who have engaged in supplemental, temporary, project- or contract-based work within the past 12 months.
- dredmorbius 10y ago
  Sampling error or bias is a hugely more significant factor than sample size. Once you're over 60 - 300 survey participants, you're well within large-number stats territory. If you're trying to estimate multiple points of freedom, you might have to increase your sample size. Simply drawing a larger sample of a biased sampling methodology isn't going to solve any problems. You'd have to aggressively correct for errors. The survey here was conducted by a firm dedicated to such work (a positive sign), but it's attached to a large public-relations firm (Edelman), which I consider a very strong negative. It isn't a disinterested research organisation or dedicated polling firm such as Pew or Harris. Edelman's other clients include Microsoft, Johnson & Johnson, Pfizer, HP, and Samsung. Quite the list of rusted stars. https://en.wikipedia.org/wiki/Edelman_(firm) https://en.wikipedia.org/wiki/Edelman_(firm)
- jomamaxx 10y ago
  The sample size is just fine, actually, it's very large. The issue is randomness. If it's a random sample, it's very large. If it's not a random sample, it's worthless. "We have no idea how they found the participants or controlled for bias in the selection process." A) This has nothing to do with sample size B) If they have any credibility whatsoever, they hopefully know what they are doing. Adjusting is a statistically legit thing to do, they do it in political polling (and in hindsight) it works. But yes - with 'online surveys', it's hard.
- jnbiche 10y ago
  > I'm aware of sample size determination, but minimum sample sizes typically assume random sampling over the target population. If you were aware of sample size determination, why would you suggest that a 6000 person sample is "pretty small"? It's pretty big for a national study, actually. Whether or not the sample selection was properly done with attention paid to bias is difficult to say, but has little to do with the sample size since a non-randomly-selected sample is likely worthless whether it's 6000 or 60,000. I have some significant concerns about this study, but sample size is not among them.
  - mdorazio 10y ago
    Specifically because their methodology for sample selection is entirely hidden. If you do a generic online survey with self-selected respondents then 6k is not very many people from which to be drawing sweeping conclusions. Edit: also, I disagree that sample size is entirely worthless in non-random sample selection. If your sample size is a big enough percentage of the target population you are studying, you can draw some conclusions regardless of randomness. You just end up having to use absurdly large samples to get meaningful data.

by objclxt 10y ago

mdorazio 10y ago

I'm aware of sample size determination, but minimum sample sizes typically assume random sampling over the target population. All their sampling methodology slide says is that it was an online survey of adults who have done some paid work in the last 12 months, and they did some weighting to match BLS stats. We have no idea how they found the participants or controlled for bias in the selection process.

markcampbell 10y ago
This is a very important aspect of the study. They did not do a true randomized survey as participants were self-selecting (online surveys have this inherently -- well, mostly).
- dredmorbius 10y ago
  It's not clear that this was or wasn't a self-selecting sample. The article does state that the surveying was conducted online: In the research, more than 6,000 U.S. working adults over the age of 18 were surveyed online by independent research firm Edelman Intelligence between July 29 — August 24, 2016. Among them, 2,049 were freelancers and 3,953 were non-freelancers. The survey defined freelancers as individuals who have engaged in supplemental, temporary, project- or contract-based work within the past 12 months.
dredmorbius 10y ago
Sampling error or bias is a hugely more significant factor than sample size. Once you're over 60 - 300 survey participants, you're well within large-number stats territory. If you're trying to estimate multiple points of freedom, you might have to increase your sample size. Simply drawing a larger sample of a biased sampling methodology isn't going to solve any problems. You'd have to aggressively correct for errors. The survey here was conducted by a firm dedicated to such work (a positive sign), but it's attached to a large public-relations firm (Edelman), which I consider a very strong negative. It isn't a disinterested research organisation or dedicated polling firm such as Pew or Harris. Edelman's other clients include Microsoft, Johnson & Johnson, Pfizer, HP, and Samsung. Quite the list of rusted stars. https://en.wikipedia.org/wiki/Edelman_(firm) https://en.wikipedia.org/wiki/Edelman_(firm)
jomamaxx 10y ago
The sample size is just fine, actually, it's very large. The issue is randomness. If it's a random sample, it's very large. If it's not a random sample, it's worthless. "We have no idea how they found the participants or controlled for bias in the selection process." A) This has nothing to do with sample size B) If they have any credibility whatsoever, they hopefully know what they are doing. Adjusting is a statistically legit thing to do, they do it in political polling (and in hindsight) it works. But yes - with 'online surveys', it's hard.
jnbiche 10y ago
> I'm aware of sample size determination, but minimum sample sizes typically assume random sampling over the target population. If you were aware of sample size determination, why would you suggest that a 6000 person sample is "pretty small"? It's pretty big for a national study, actually. Whether or not the sample selection was properly done with attention paid to bias is difficult to say, but has little to do with the sample size since a non-randomly-selected sample is likely worthless whether it's 6000 or 60,000. I have some significant concerns about this study, but sample size is not among them.
- mdorazio 10y ago
  Specifically because their methodology for sample selection is entirely hidden. If you do a generic online survey with self-selected respondents then 6k is not very many people from which to be drawing sweeping conclusions. Edit: also, I disagree that sample size is entirely worthless in non-random sample selection. If your sample size is a big enough percentage of the target population you are studying, you can draw some conclusions regardless of randomness. You just end up having to use absurdly large samples to get meaningful data.