5 ms·

To improve my criticism (open to correction, especially because I don't have a background in academic psychology), I hypothesize that the paper's results don't

To improve my criticism (open to correction, especially because I don't have a background in academic psychology), I hypothesize that the paper's results don't apply and can't be generalized to the broader U.S. population. For Study 1, I specifically think that one's views on friendship are shaped by the people around them (so the views of undergraduates on friendship in a small Northeastern college might be different than a large state school in Texas or California). Views on friendship may also change after graduating university (where it's harder to make friends), and may also be different than views from people who have never attended (e.g. people in small towns in a trade or who have spent a career enlisted in the military). For Studies 2-3 (this is likely where my reasoning is shakiest), I hypothesize that people on Mechanical Turk represent a small subset of the U.S. population, and most have specific shared beliefs (that motivate them to trade time doing fairly simple tasks for money). It's possible that some people are on Mechanical Turk for fun or specifically to learn more about research by participating in online studies, but I hypothesize that this is a negligible part of the population. In short, I'm not convinced that the study's design makes its conclusions applicable to most women and men in the United States population.

xapata 5y ago
The fact that a sample is a tiny part of the population is irrelevant. The question is how that sample differs from the population in a way that correlates with the explained/dependant variable, but is not included as an explanatory/independent variable. Your point that views on friendship may change over time is a strong criticism. The criticism of the Turk selection is weak as written. How does spending time on Turk relate to friendship preferences?
- openknot 5y ago
  To clarify the rationale: I have a hunch that men spending time on Mechanical Turk are either less financially secure or care more about maximizing the money one earns from spare time. This may cause these men to prefer male friends who can help them make more money or raise their social status to help them find better work. Therefore, a sample of men who do not need to worry about money (and thus do not feel a want to use Mechanical Turk) may not feel a need to choose friends based on their usefulness to their financial or social status, and instead prefer male friends who are trustworthy and provide a higher level of emotional support. My hunch is backed up by this segment of a review paper from 2017 [0]: "Further, half of the MTurkers are employed part-time or unemployed. This is consistent with past findings (e.g., Ross et al., 2010; Behrend et al., 2011). A population of unemployed or underemployed participants offers a potentially interesting sampling opportunity for research questions related to job search behavior, underemployment, and unemployment (Roulin, 2015). At the same time, this could explain why MTurkers have a lower income on average as compared to nationally representative samples. "Taken together, our conclusions echo the conclusions of other researchers (cf., Behrend et al., 2011; Buhrmester et al., 2011; Berinsky et al., 2012; Casler et al., 2013; Brandon et al., 2014; Woo et al., 2015): MTurk samples are not representations of the general population." [0] From "Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations" published in the Frontiers in Psychology journal: https://doi.org/10.3389/fpsyg.2017.01359 https://doi.org/10.3389/fpsyg.2017.01359
  - xapata 5y ago
    I'm convinced :-) Despite the paper's conclusion sounding nice, like a horoscope, the evidence seems more anecdote than data. I'd call it a "just so" story. When everyone likes the conclusion, they tend to banish their skepticism.

xapata 5y ago

The fact that a sample is a tiny part of the population is irrelevant. The question is how that sample differs from the population in a way that correlates with the explained/dependant variable, but is not included as an explanatory/independent variable. Your point that views on friendship may change over time is a strong criticism. The criticism of the Turk selection is weak as written. How does spending time on Turk relate to friendship preferences?

openknot 5y ago
To clarify the rationale: I have a hunch that men spending time on Mechanical Turk are either less financially secure or care more about maximizing the money one earns from spare time. This may cause these men to prefer male friends who can help them make more money or raise their social status to help them find better work. Therefore, a sample of men who do not need to worry about money (and thus do not feel a want to use Mechanical Turk) may not feel a need to choose friends based on their usefulness to their financial or social status, and instead prefer male friends who are trustworthy and provide a higher level of emotional support. My hunch is backed up by this segment of a review paper from 2017 [0]: "Further, half of the MTurkers are employed part-time or unemployed. This is consistent with past findings (e.g., Ross et al., 2010; Behrend et al., 2011). A population of unemployed or underemployed participants offers a potentially interesting sampling opportunity for research questions related to job search behavior, underemployment, and unemployment (Roulin, 2015). At the same time, this could explain why MTurkers have a lower income on average as compared to nationally representative samples. "Taken together, our conclusions echo the conclusions of other researchers (cf., Behrend et al., 2011; Buhrmester et al., 2011; Berinsky et al., 2012; Casler et al., 2013; Brandon et al., 2014; Woo et al., 2015): MTurk samples are not representations of the general population." [0] From "Systems Perspective of Amazon Mechanical Turk for Organizational Research: Review and Recommendations" published in the Frontiers in Psychology journal: https://doi.org/10.3389/fpsyg.2017.01359 https://doi.org/10.3389/fpsyg.2017.01359
- xapata 5y ago
  I'm convinced :-) Despite the paper's conclusion sounding nice, like a horoscope, the evidence seems more anecdote than data. I'd call it a "just so" story. When everyone likes the conclusion, they tend to banish their skepticism.