3 ms·

I see no one has mentioned Byte-Pair Encoding (BPE) yet. https://www.gwern.net/GPT-3#bpes https://www.gwern.net/GPT-3#bpes The reason ChatGPT struggles with t

I see no one has mentioned Byte-Pair Encoding (BPE) yet. https://www.gwern.net/GPT-3#bpes https://www.gwern.net/GPT-3#bpes The reason ChatGPT struggles with tasks involving individual character counting is that it can't see individual characters. The word "prime" is a single token (https://beta.openai.com/tokenizer https://beta.openai.com/tokenizer) to the eyes of ChatGPT, and 1 is not prime.

layer8 4y ago
That doesn’t explain why it still provides two answers (getting the number of letters right despite tokenization) and also doesn’t notice that they are contradictory.
- 152334H 4y ago
  Forgive the anthropomorphising, but I find it very similar to how I'd handle someone forcing me to explain two contradictory claims coherently.
  - layer8 4y ago
    But no one was forcing it to do that, and it doesn’t seem it has any awareness that it could proceed differently.
chatgptexpert 4y ago
This doesn't seem like a good explaination. If you ask ChatGPT to separate each letter in a word with a comma, it will do it correctly. So it can "see" and do operations at the character level.