4 ms·
>It's worth digging into the methodology here: the study participants were found through Mechanical Turk That seems to be most studies these days. And 10 years
by weberer 4y ago
>It's worth digging into the methodology here: the study participants were found through Mechanical Turk
That seems to be most studies these days. And 10 years ago it was mostly "we've recruited 200 students from [university performing the study]" and people complained about that too. And if you do a phone survey, then its "you're biasing toward the people who pick up calls from unknown numbers". There's always going to be some sort of sampling bias, and this study doesn't seem to be any sort of special outlier.
- sejje 4y ago> There's always going to be some sort of sampling bias I don't agree, I think we'll get pretty good at this. I agree there currently is pretty much always a sampling bias, though. Maybe the exception is when the study samples a particular group already, rather than the population at large.
- oh_my_goodness 4y agoI agree, but unfortunately that doesn't address the complaint. If surveys are in fact whacked out by bias (idk), then we won't get certain kinds of meaningful data from those surveys. It's tough for the surveyors, but that doesn't overcome the objection. I think in some fields this may be more obvious. If particles are too big for the LHC to find them, then we won't get meaningful data about those particles from those experiments. We can point out that we built a ridiculously, unbelievably huge collider and what do people expect? It's a valid point, but too bad for us.
- naniwaduni 4y agoMaybe we could fund the IRS to to conduct more audits by funneling them the grant money that otherwise funds mturk survey "research"! (This definitely seems like a great idea and nothing can possibly go wrong)