3 ms·
Current LLMs are not aware of the letters in text. It's too expensive computationally to assign a token to each letter. They operate at the level of tokens, w
by retrac 2y ago
Current LLMs are not aware of the letters in text. It's too expensive computationally to assign a token to each letter. They operate at the level of tokens, which tend to be word-level: each morph-em-e is broke-n down in-to seg-ment-s and assign-ed an int-e-ger.
The result is that LLMs are quite oblivious to spelling and such. There are some exceptions working at the letter level, but mostly, yeah. This applies to ChatGPT etc.
- PaulHoule 2y agoMost LLMs can't even talk about the tokenization they use which I think indicates that they don't have the right knowledge to do wordplay at the letter level.