Slightly offtopic, but I am the author of the Rails/Mechanical Turk integration gem, Turkee ( http://www.github.com/aantix/turkee http://www.github.com/aantix/t

Slightly offtopic, but I am the author of the Rails/Mechanical Turk integration gem, Turkee ( http://www.github.com/aantix/turkee http://www.github.com/aantix/turkee ). The lower the cost of the HIT (and the easier the question) the more you want to make sure you're screening for bots. Make sure you have a gold standard question. For every HIT ask two questions; ask the legitimate question that you need answered and then on top of that, ask a simple "gold-standard" question that will determine whether the answer submitted is a bot or not. For the gold standard, I like to choose two numbers, and store the sum of their values. Along with your HIT question, be sure to ask the Turker the value of their sum. E.g. For a survey you ask the Turker "What is your favorite color?" But then along with that, you ask them "What is four plus fve?" Be sure to misspell one value (e.g. remove a letter, etc). If the Turker doesn't respond correctly to the "gold standard" question, programmatically reject their response.