The smaller the tokens the more guesses the LLMs need for generating text. LLMs are really good at guessing, but the more they have to guess the likelier for th

The smaller the tokens the more guesses the LLMs need for generating text. LLMs are really good at guessing, but the more they have to guess the likelier for them to lose their train off thought.