3 ms·
Tokeninazation make this kind of “dyslexia” very unlikely
by jerpint 3y ago
Tokeninazation make this kind of “dyslexia” very unlikely
- minimaxir 3y agoSometimes yes, but in this case you can verify that in the phrase "gpt-3.5-turbo", the only doubled up characters are "gpt-3.5-turbo", so the number can still be wrong by swapping a single token. https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer