3 ms·List of all 100k GPT-4 tokens3 points by s-macke 3y agoneximo64 3y agoHow do you get these?s-macke 3y agoExtracted via the GitHub Repo tiktoken [1] After you try to decode a string the list on my computer shows up in /tmp/data-gym-cache/9b5ad71b2ce5302211f9c61530b329a4922fc6a4 [1] https://github.com/openai/tiktoken https://github.com/openai/tiktoken