5 ms·
https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer or the official python library tiktoken https://github.com/openai/tiktoken https://g
by sp332 3y ago
https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer or the official python library tiktoken https://github.com/openai/tiktoken https://github.com/openai/tiktoken or this JS port of tiktoken https://github.com/dqbd/tiktoken https://github.com/dqbd/tiktoken
- oofsa 3y agohttps://platform.openai.com/tokenizer https://platform.openai.com/tokenizer is not for GPT-4 but GPT-3. https://tiktokenizer.vercel.app/ https://tiktokenizer.vercel.app/ supports GPT-4.
- adroitboss 3y agoThanks for the recommendations. I found the one I needed because of this comment thread.