2 ms·
Unfortunately, in CJK languages, it averages to slightly more than one token per character.
by glandium 3y ago
Unfortunately, in CJK languages, it averages to slightly more than one token per character.
- TeMPOraL 3y agoI suppose we have a quite objective way of measuring and comparing semantic density of natural languages.