3 ms·
Could you please elaborate on what you mean by "intelligently validate the quality of labels in a document and complement human judgment", and discuss your meth
by staticautomatic 7y ago
Could you please elaborate on what you mean by "intelligently validate the quality of labels in a document and complement human judgment", and discuss your methodology?
This seems to operate under the assumption that human labels are not actually the ground truth. I understand that they can be dirty, but most unsupervised approaches aren't producing a ground truth, either. So, are you saying it's better to have multiple pretty good sources of truth instead? Because depending on the application, that might make sense or it might be like trying to start a farm with a dead horse and a dead cow.
- flyx 7y agoCertainly. Our philosophy is to complement human wisdom with computer precision. Humans may often be labeling for 8 hours a day and may get fatigued. So if Starbucks has been labeled as a cafe 35x in a document and as a person 2x, we can flag this and ask "hey, are you sure you wanted to label this as a person?". Or if we know for a fact Canada is a country, but it's labeled as an animal in a document, we can raise a flag as well. This won't work for everything, but we think it can help with quality assurance.