5 ms·
I have long suspected that Turkers dishonestly perform the tasks. At 0.06 cents per task, you're really incentivizing "finish the task as quickly as possible";
by CGamesPlay 4y ago
I have long suspected that Turkers dishonestly perform the tasks. At 0.06 cents per task, you're really incentivizing "finish the task as quickly as possible"; and "press the left button" is a lot faster than "read the tweet, think about it, and classify".
- ftxbro 4y ago> At 0.06 cents per task Do you work at Verizon?
- CGamesPlay 4y agoI got that figure from https://news.ycombinator.com/item?id=35335558 https://news.ycombinator.com/item?id=35335558 [append] Oh, it's the xkcd Verizon agent reference. Got it. My bad.
- ftxbro 4y agoI was nitpicking about 0.06 dollars versus 0.06 cents. It's this ancient meme, where a guy records a customer service call with Verizon where they have some back-and-forth about whether these are the same or not. https://www.youtube.com/watch?v=MShv_74FNWU https://www.youtube.com/watch?v=MShv_74FNWU
- renewiltord 4y agoYou always use multiple MTurks on same job when you use them.
- CGamesPlay 4y agoOK, but if all MTurks are doing this (because why wouldn't they) then you are just sampling a random variable.
- johnthewise 4y agoYou are not just taking the average, answers would have to be consistent with each other.
- dotancohen 4y agoAnd you'll thus see that the answers converge at the rate a random variable would. Whereas with more accurate methods, the answers will generally agree with one another.