3 ms·

How does that fare for non-English queries? E.g. are two chars still not enough for Chinese languages?

• meepmorp 3y ago
  I've seen 1.5 chars/token used as a rule of thumb for estimating token counts in Chinese text.