Wasn't one of the important properties of tokens that they allow the LLM to represent a sort of "semantic unit"? I.e. If you generate an embedding over words, t

Wasn't one of the important properties of tokens that they allow the LLM to represent a sort of "semantic unit"? I.e. If you generate an embedding over words, the embedding can capture meaningful semantic relations between words, because most words do have a relatively narrowly-defined meaning. In contrast, an embedding over letters/characters/bytes/etc can't do the same because the meaning of those depends entirely on context. I thought, tokens were sort of a compromise between the two: they still allowing meaningful embeddings for frequent, "large" tokens while also including "small" tokens that can be used to make up new words.