5 ms·
Interesting that MIT Tech Review wrote an article on this, since a lot of great research (at various universities, including MIT) is now a lot easier due to eas
by rdl 14y ago
Interesting that MIT Tech Review wrote an article on this, since a lot of great research (at various universities, including MIT) is now a lot easier due to easily available human task workers. Hopefully this will make research easier, maybe reduce grad-student drudgery, and speed up scientific progress for everyone. (Task work is useful for business, too, but it seems particularly useful in the same ways cloud computing is useful -- the ability to rapidly spin up a short-term task to handle load, without persistent staffing costs.)
- prayag 14y ago>since a lot of great research (at various universities, including MIT) is now a lot easier due to easily available human task workers This is very true. A lot of researchers today are using MobileWorks and other crowdsourcing platforms. It has a lot of implication in a number of research areas including AI, HCI, Databases, Bio-sciences, Earth-Sciences and a lot more. These are all fields that have used crowdsourcing and have published papers about crowdsourcing. In fact, MobileWorks came out of UC Berkeley where the founders were doing research in HCI using crowdsourcing before we started building our own platform because we thought the existing competitors were a pain to use.
- dchichkov 14y agoUnlike turkers, grad students produce much cleaner data and also see the data first hand. Which is important. I consider mturk/offshore as a necessary evil, rather than a panacea that will "make research easier and speed it up". If data quality matters and you can avoid mturk - avoid it.
- prayag 14y ago>Unlike turkers, grad students produce much cleaner data and also see the data first hand. Which is important. I consider mturk/offshore as a necessary evil, rather than a panacea that will "make research easier and speed it up". This is what we are trying to solve. You can get good clean data from crowdsourcing too if you give it out to the most suitable workers and have them work collaboratively.
- anandkulkarni 14y ago>If data quality matters and you can avoid mturk - avoid it. You've identified a great reason to use MobileWorks instead. Crowd work doesn't have to be low-quality; Turk just hasn't done a terrific job of it. We've achieved substantially higher quality than MTurk by treating workers better, paying fairer wages, and giving workers tools to collaborate.
- huahaiy 14y agoYou get what you pay for. If you pay them pennies, you get garbage, but if you pay them equitable wages, you get real work done. It's that simple.
- aantix 14y agoSeems like you're painting with a broad brush. Design of HITs on Mechanical Turk is crucial. Implement gold standard questions per assigment to ensure that you're not receiving spam from a bot.
- anandkulkarni 14y agoIt turns out this is one of the core problems with Mechanical Turk that drives folks to MobileWorks instead. In Turk, users spend so much time implementing protection against malicious workers and bots that they need to develop real expertise in crowdsourcing in order to be able to outsource their work effectively. We've filtered workers more effectively up front and reduced the incentive to cheat substantially, so things largely just work.