2 ms·
> The disdain for the qualitative expertise of domain experts who produce data is a well-understood guilty secret within ML circles, embodied in Frederick Jelin
by thaumasiotes 5y ago
> The disdain for the qualitative expertise of domain experts who produce data is a well-understood guilty secret within ML circles, embodied in Frederick Jelinek’s ironic talk, "Every time I fire a linguist, the performance of the speech recognizer goes up."
This reminds me of how the chimp sign language studies got much better results from hearing evaluators than from deaf ones.
- pjscott 5y agoDoctorow seems to be missing the meaning of that quote (which is also not the title of a talk, ironic or otherwise). It was specifically a comment on the usefulness of computer language models created manually based on linguistic theories of grammar versus ones in the same model family created automatically from real-world data -- the latter tended to work better. These days I usually hear it quoted more broadly as a warning about the danger of encoding too much possibly-wrong domain knowledge in an ML system when a more generic model and the training data are sufficient to learn the useful parts on their own. Neither of those translates into disdain for qualitative understanding of the underlying reality behind the data set, which is one of those things that everyone knows is important. The problem is that such understanding is actually hard, and easy to mess up even when you're trying.