3 ms·
> but the reality is that people spend more time talking about a product they want to make than it takes to make enough labels yourself And what might the vali
by jruohonen 3y ago
> but the reality is that people spend more time talking about a product they want to make than it takes to make enough labels yourself
And what might the validity of such self-made labels be, if I may ask?
- PaulHoule 3y agoDepends on what you are labeling. How accurately could you look at pictures of cars and tell marked police cars from regular cars? When I do them in a hurry though I fat finger maybe 1-5% of them so it takes some cleanup.
- jruohonen 3y agoFair enough, and that's why I suppose for instance CATPCHAs and such make us do silly free labor. But generally, hilarious labeling errors are widespread already in benchmarks: https://news.ycombinator.com/item?id=26628778 https://news.ycombinator.com/item?id=26628778 Companies also seem rather carefree with their labeling, including even in contexts in which accuracy is paramount: https://news.ycombinator.com/item?id=38455338 https://news.ycombinator.com/item?id=38455338 And things get interesting when you start labeling alleged political bias, for instance: https://news.ycombinator.com/item?id=35982799 https://news.ycombinator.com/item?id=35982799 Thus, in general, I'd appreciate research on the "accuracy" (i.e., labelers' inter-group agreement, etc.) of labels used in the wild.
- canaus 3y agoHow much more useful would it be to get multiple people to label data?