3 ms·
>20 tokens per second (~4 words per second) How can there be 5 tokens per word, when they have more than half the vocabulary as GPT-2/3 which has 1.3 tokens pe
by sebzim4500 4y ago
>20 tokens per second (~4 words per second)
How can there be 5 tokens per word, when they have more than half the vocabulary as GPT-2/3 which has 1.3 tokens per word?
I would have guessed more like 1.5 tokens per word.
- fwlr 4y agoOh, it’s probably higher than four words per second, then. I assumed tokens was characters and used the standard “there are five characters in a word” rule of thumb.
- MacsHeadroom 4y agoIt's about 4 charcters per token. So just over 1 token per word. I just round to 1 token per word since text most people generate does not use larger words and because larger common words are still encoded as one token (e.g. HackerNews is probably one token despite being 10 characters).
- Taek 4y agoI typically see people claim 2-3 tokens per word.