4 ms·
https://twitter.com/sherwinwu/status/1603522777968832512 https://twitter.com/sherwinwu/status/1603522777968832512 appears to include it.
by bryanh 4y ago
https://twitter.com/sherwinwu/status/1603522777968832512 https://twitter.com/sherwinwu/status/1603522777968832512 appears to include it.
- minimaxir 4y agoSo it does. The code there implies cl100k_base has a vocab size of 100k (I guess it's in the name lol) which means it is more comprehensive than GPT-2's 50k, so fewer tokens will be necessary.