2 ms·
Definitions are not enough to fully capture the meaning of a word. In order to do that you need full language modelling and to ground words into other sensory m
by visarga 8y ago
Definitions are not enough to fully capture the meaning of a word. In order to do that you need full language modelling and to ground words into other sensory modalities, plus the word in relation to actions taken in various situations when the word was used.
GPT-2 (of recent OpenAI fame) uses 1.5 billion parameters and, though capable of interesting results, is far from human level. It also uses just text so it's incomplete.
https://blog.openai.com/better-language-models/ https://blog.openai.com/better-language-models/
Another interesting metric is Bits Per Character - BPC. The state of the art is around 1.06 on English Wikipedia. This measures the average achievable compression on character sequences and doesn't include the size of the model, just the size of the compressed sequence.
https://arxiv.org/pdf/1808.04444.pdf https://arxiv.org/pdf/1808.04444.pdf
- Emma_Goldman 8y agoThat's true but it's almost inherent in what a dictionary is, i.e. to catalogue the canonical semantic meaning of words, not to provide a complete model of language and its contextual variables.