4 ms·
Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc
by KhoomeiK 4y ago
Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc as input, models seem to learn something like syntax.
https://arxiv.org/abs/1905.05950 https://arxiv.org/abs/1905.05950
https://aclanthology.org/N19-1419/ https://aclanthology.org/N19-1419/
https://arxiv.org/abs/1906.04341 https://arxiv.org/abs/1906.04341
- compressedgas 4y agoI would add https://www.pnas.org/doi/10.1073/pnas.1907367117 https://www.pnas.org/doi/10.1073/pnas.1907367117
- KhoomeiK 4y agoYeah this is more or less a re-release of the second work I linked, except for a broader audience and more targeted at linguistics / cognitive science researchers.