3 ms·

Can anyone comment on how common it is for social psychology research to rely on surveys completed by Turkers?

tw1010 8y ago
You probably shouldn't incorperate a scientific study into your model of the world unless you've also memorized the methodology that it's attached to and relies on.
mrtksn 8y ago
According to Duncan Watts in his book "Everything is Obvious, once you know it" it's quite widespread to use the Turkers for that kind of research. It's described as a very cost-effective method compared to reaching out the subject directly and trying to convince them to gather somewhere for the study. Also, it's described as game-changing when it comes to studies that must have a large base and made previously impossible(due to practical reasons) studies possible.
- JamesCoyne 8y ago
  Thanks.
neuromantik8086 8y ago
When I graduated from undergrad, Mechanical Turk was just starting to make inroads in social psych research. One of my friends was actually one of the first students to manage a survey for the social psych professor using MTurk. It's been promoted relatively heavily at conferences since. It's been discussed here [0] as well, but it's important to highlight that prior to MTurk social psychology studies were (and actually probably still are to a great extent) widely susceptible to the "college sophomore problem" wherein a sample meant to provide conclusions that would generalize to the wider population of the human race was composed almost exclusively of college age children. This meant that a large number of conclusions in social psychology have historically been based on non-representative samples. Methodologically, MTurk can be seen as a major improvement over the status quo in this regard. [0] https://news.ycombinator.com/item?id=17789595 https://news.ycombinator.com/item?id=17789595
- whorleater 8y ago
  apparently 40-50% of social sciences surveys were conducted through mturk by 2016: https://doi.org/10.1037/pspa0000056 https://doi.org/10.1037/pspa0000056

You probably shouldn't incorperate a scientific study into your model of the world unless you've also memorized the methodology that it's attached to and relies on.

mrtksn 8y ago

According to Duncan Watts in his book "Everything is Obvious, once you know it" it's quite widespread to use the Turkers for that kind of research. It's described as a very cost-effective method compared to reaching out the subject directly and trying to convince them to gather somewhere for the study. Also, it's described as game-changing when it comes to studies that must have a large base and made previously impossible(due to practical reasons) studies possible.

JamesCoyne 8y ago
Thanks.

neuromantik8086 8y ago

When I graduated from undergrad, Mechanical Turk was just starting to make inroads in social psych research. One of my friends was actually one of the first students to manage a survey for the social psych professor using MTurk. It's been promoted relatively heavily at conferences since. It's been discussed here [0] as well, but it's important to highlight that prior to MTurk social psychology studies were (and actually probably still are to a great extent) widely susceptible to the "college sophomore problem" wherein a sample meant to provide conclusions that would generalize to the wider population of the human race was composed almost exclusively of college age children. This meant that a large number of conclusions in social psychology have historically been based on non-representative samples. Methodologically, MTurk can be seen as a major improvement over the status quo in this regard. [0] https://news.ycombinator.com/item?id=17789595 https://news.ycombinator.com/item?id=17789595

whorleater 8y ago
apparently 40-50% of social sciences surveys were conducted through mturk by 2016: https://doi.org/10.1037/pspa0000056 https://doi.org/10.1037/pspa0000056