4 ms·
Cool experiment. Thinking of using Turk for photo approvals on my site, but the accuracy issue leaves me slightly hesitant. Has anyone had experience doing this
by ashishk 17y ago
Cool experiment. Thinking of using Turk for photo approvals on my site, but the accuracy issue leaves me slightly hesitant. Has anyone had experience doing this?
- eru 17y agoFrom what I read the standard technique seems to be let two people review everything. When there's a discrepancy you can add more manpower (and take the majority view or so) or look at it yourself.
- eli 17y agoYup. There are a couple of language translation services that use techniques like that.
- jules 17y agoA problem is that there are rooms of people turking together. They'd probably all submit the same translation, for example.
- eru 17y agoI guess you should flag exactly similar translations. Though I do not see how you can do this without relying on security by obscurity.
- jfno67 17y agoWhat I did was to have up to five person work on a hit. Many simple hits are done at 0.05$ so it's still not too expansive. For text submissions, you can enforce validation rules (simple and regex). So you don't have to manually reject non-conforming work. You can also have the Turker fill in their work on a webpage that you control, so validation can be much more advanced. I never went all the way but sometimes I think about it.
- hyung 17y agoHey ashishk, sorry for using HN to contact you, but I can't seem to find your contact info anywhere. Our company provides very affordable and accurate photo approval services. Photos are manually approved and we can handle anywhere from 10-1,000,000 photos per day. If you're interested, feel free to email me (email address is in my profile).