4 ms·
I'm impressed with the accuracy of MobileWorks over mechanical Turk (or pure MT- not sure if MW just filters workers heavily). I tried a receipt transcription t
by physcab 14y ago
I'm impressed with the accuracy of MobileWorks over mechanical Turk (or pure MT- not sure if MW just filters workers heavily). I tried a receipt transcription task where I fed the same instructions and same photo to both services. Out of the box, MW was 100% while MT was about 90% for the same cost.
- anandkulkarni 14y agoGlad to hear it! This is an important philosophical difference for us. Quality control in a cloud labor service should be the platform's job, not the customer's. The system should work correctly out of the box. And yes, our workforce is separate from Turk's - part of the reason the quality is better.
- prayag 14y agoAs Anand said we have a separate workforce. We have a bunch of algorithmic and social elements that we have developed for improving quality. In fact,we have written an IEEE paper on how to improve quality in crowdsourcing. http://www.cs.berkeley.edu/~bjoern/papers/kulkarni-mobileworks-ieee2012.pdf http://www.cs.berkeley.edu/~bjoern/papers/kulkarni-mobilewor...