3 ms·

Text encodes in fewer tokens, and language coverage is better.

• lalaithion 3y ago
  I understand the theory, I was looking for an example of the same text tokenized with the two different vocabularies.