3 ms·
Yeah, see [1]. [1] https://github.com/openai/tiktoken/blob/main/tiktoken/model.py https://github.com/openai/tiktoken/blob/main/tiktoken/model....
by lifthrasiir 3y ago
Yeah, see [1].
[1] https://github.com/openai/tiktoken/blob/main/tiktoken/model.py https://github.com/openai/tiktoken/blob/main/tiktoken/model....