2 ms·
You can see it by yourself here if you are interested: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer The tokens are usually not m
by rvnx 3y ago
You can see it by yourself here if you are interested: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
The tokens are usually not matching the length of a word