4 ms·
Tools for exploring the models are already being developed. Language models that are not given concepts of part-of-speech, dependency trees, grammars, develop
by knuthsat 6y ago
Tools for exploring the models are already being developed.
Language models that are not given concepts of part-of-speech, dependency trees, grammars, develop representations of their own and people have found ways to inspect them.
- Schiphol 6y agoCould you please provide pointers to relevant papers? I'm very interested.
- desku 6y agoHere's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950 https://arxiv.org/abs/1905.05950