Seconded. Quality control is a nightmare with Turk, often requiring stimuli to be labeled multiple times and have a variety of judging approaches to "crown a w

Seconded. Quality control is a nightmare with Turk, often requiring stimuli to be labeled multiple times and have a variety of judging approaches to "crown a winner". Companies like DefinedCrowd (https://www.definedcrowd.com/ https://www.definedcrowd.com/) have taken a quality-first approach which gives much better data, but of course, at a cost.