3 ms·
Tiny models can already learn language (as in grammar and syntax) from very little very quickly. It's the semantics that separates LLMs from CharRNN.
by lostmsu 3y ago
Tiny models can already learn language (as in grammar and syntax) from very little very quickly.
It's the semantics that separates LLMs from CharRNN.
- GolDDranks 3y agoThat's interesting! You mention CharRNN. Any links? I find it especially interesting to test whether models that know syntax, know superficially human language -like syntax, or actually syntax that conforms to the deep (syntactic) structure of human languages.
- lostmsu 3y agohttps://github.com/karpathy/char-rnn https://github.com/karpathy/char-rnn