3 ms·
> We have extended the model’s context length from 128k to 1M, which is approximately 1 million English words Actually English language tokenizers map on avera
by lr1970 2y ago
> We have extended the model’s context length from 128k to 1M, which is approximately 1 million English words
Actually English language tokenizers map on average 3 words into 4 tokens. Hence 1M tokens is about 750K English words not a million as claimed.
- deleted 2y ago[deleted]
- swyx 2y agogood, its been hours since i saw a "well actually" comment on HN