3 ms·
Text is broken into something called "tokens" [0] for GPT. It doesn't work on individual characters. That explains why it always fails these character counting
by xaphod 4y ago
Text is broken into something called "tokens" [0] for GPT. It doesn't work on individual characters. That explains why it always fails these character counting questions.
[0]: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-...
- charcircuit 4y ago>it always fails It usually is able to get the answer within 1 character of the actual answer.
- nfcampos 4y agoInterestingly both “halves” of the word as it split it have the same number of tokens