> LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Although this is mostly true, there are

> LLMs work on tokens, which, for the most part, are words (longer words are generally split up into multiple tokens). Although this is mostly true, there are exceptions to the rule. Look at the ByT5 family[1] of models for instance. These models operate on byte level representations of text and are inherently token free. I read a recent paper[2] that found that replacing token based text encoders in text to image models with these byte level models improves the spelling abilities of the txt2img models (which are otherwise notoriously bad at spelling in all human languages). [1]: https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 [2]: https://arxiv.org/abs/2212.10562 https://arxiv.org/abs/2212.10562