3 ms·
It sees embeddings that is trained to encode semantic meanings. The way we tokenize is just a design choice. Character level models(e.g. karpathy's nanoGPT) ex
by isaacfung 2y ago
It sees embeddings that is trained to encode semantic meanings.
The way we tokenize is just a design choice. Character level models(e.g. karpathy's nanoGPT) exist and are used for educational purpose. You can train it to count number of 'r' in a word.
https://x.com/karpathy/status/1816637781659254908?lang=en https://x.com/karpathy/status/1816637781659254908?lang=en