5 ms·
Poll: What experiment to run on AWS mechanical turk?
After our experience using the mechanical turk we've become a bit obsessed with trying to find a really awesome use case for the service - even if its just experimental.
We have some downtime right now and we are narrowing down the experiment between the following:
1. Some sort of cooperative art project - inspired by http://www.tenthousandcents.com
2. A massive chess game that forks every time a move is made (probably a poor idea)
3. Some implementation of the Prisoner's dilemma game.
4. Any time of game to get a sense of how risk seeking or risk averse the Turks are - possibly a tie in with the Monty Hall Problem.
Most of these ideas are centered around game theory but we are pretty much open to anything (within reason). Any votes or opinions on what would be the most interesting/entertaining?
-Thanks
- zitterbewegung 18y agoHow about creating an experiment and designing it right on turk? You create a rough sketch and then you auto-correct it using people?
- antigravity 18y agoYou mean keep tinkering with the study until it "works"? If you're not careful, you could get http://en.wikipedia.org/wiki/Publication_bias http://en.wikipedia.org/wiki/Publication_bias on a massive new scale... It would be interesting to try to find scientifically optimal ways of doing study iteration, or approaches to interpreting the complete (experiment_rev, results)-list from another researcher...
- AlexeyMK 18y agoI'd love to see a prisoner's dilemma experiment via MTurk. These tend to be financial in nature, though; can you set variable payment rates per task, depending on external factors?
- sigstoat 18y agoyou can pay out arbitrary bonuses once they've answered. so you could easily do single-round prisoner's dilemma, and have amazon pair up 2 (different) folks at a time, and when you get the results back, look to see what the folks in each pair did, and award bonuses appropriately. i'm not sure if there is a way to do an iterated version.
- shafqat 18y agoWhat was your previous experience?
- adatta02 18y agowe used the Turk service to basically do OCR on about 3k images - http://shout.setfive.com/2008/11/20/artificial-artificial-intelligence-our-experiance-with-mechanical-turk/ http://shout.setfive.com/2008/11/20/artificial-artificial-in...
- iamelgringo 18y agoA friend last week told me about a business he's working with. They do Natural Language Processing to scan thousands of articles about a specific topic/product to see if they are positive, negative or neutral. They are then hoping to turn around and sell the business intelligence. I was wondering how it would compare on a cost/effectiveness basis with the Turk. Let us know what your experiment is and how you fared.
- andr 18y agoLiar's poker. Ask 2 people to pick a number and the one that picks the lower doesn't get paid. The number of people that actually play the game is the result of the experiment.
- eru 18y agoHow about a dollar auction?
- Deadsunrise 18y agoSome months ago I tried to set up a project to transcribe Alexander Shulgin's Lab books but Mturk wasn't available outside US. http://www.erowid.org/library/books_online/shulgin_labbooks/ http://www.erowid.org/library/books_online/shulgin_labbooks/
- MaysonL 18y agoSee if you can use MT to provide the fitness function for some genetic algorithm, evaluate the results, tune it, and profit!
- lacker 18y agoGo meta - use Mechanical Turk to ask what would be a really cool idea to run using Mechanical Turk.
- bjclark 18y agoI'd pay $.05 to get the them to up vote my site for certain search terms on Google's new "Search Wiki". See what happens.