3 ms·

Their test subjects came from Mechanical Turk. Is that considered reliable? I'm skeptical. And if they were using Mechanical Turk, why didn't they get 1000 s

by imp 18y ago

Their test subjects came from Mechanical Turk. Is that considered reliable? I'm skeptical. And if they were using Mechanical Turk, why didn't they get 1000 subjects instead of just 25?

weaksauce 18y ago
I agree that Mechanical Turk is a huge red flag as far as quality goes, but 1000 subjects looking at 6,821 loan applications could get fairly expensive. That is 6,821,000 *(number of questions per loan app and it looks like seven factors just from the article) HITS. This could be fairly costly depending on the price that the researchers set per question.
ars 18y ago
I think you got it backward, read it again. He used the Mechanical Turk to put a number on each face rather than just yes/no. He then compared the results with actual loan applications, and found that it correlated pretty well.
- imp 18y ago
  Oh, I see now. Makes sense. Thanks.
  - jeffbarr 18y ago
    There's some clarification on how the Turk was used at http://www.econsteve.com/?p=166 http://www.econsteve.com/?p=166 .
    - imp 18y ago
      Nice find!
      - jeffbarr 18y ago
        Uh yeah, the blogger is my son.
        imp 18y ago
        Oh, cool. I was wondering if there was a connection.