4 ms·
How do they handle bogus results from MTurk? I know people that tried to use it and had to stop because 95% of the "workers" just put meaningless data in hoping
by vultour 8y ago
How do they handle bogus results from MTurk? I know people that tried to use it and had to stop because 95% of the "workers" just put meaningless data in hoping you don't notice and they get paid.
- JimmyAustin 8y agoIt depends on the task, but if you can get them to do multiple tasks in one, you can ask them to do a task you already know the answer to. If they get the known answer wrong, instant reject.
- PretzelFisch 8y agoCouldn't you just solve that by sending the job to multiple workers and comparing the results?
- shanghaiaway 8y agoThink about it.
- deleted 8y ago[deleted]
- vultour 8y agoThat's an obvious solution, but you might start approaching a point where it's actually cheaper to hire a dedicated person for it.
- srtjstjsj 8y agoThat doesn't solve the problem of a person being wrong.