I agree that Mechanical Turk is a huge red flag as far as quality goes, but 1000 subjects looking at 6,821 loan applications could get fairly expensive. That is

I agree that Mechanical Turk is a huge red flag as far as quality goes, but 1000 subjects looking at 6,821 loan applications could get fairly expensive. That is 6,821,000 *(number of questions per loan app and it looks like seven factors just from the article) HITS. This could be fairly costly depending on the price that the researchers set per question.