4 ms·

Why does the tokenization matter? Also, ChatGPT doesn't "see" those numbers, it just sees smaller binary strings, and a human could add that series of binary nu

by rcme 3y ago

Why does the tokenization matter? Also, ChatGPT doesn't "see" those numbers, it just sees smaller binary strings, and a human could add that series of binary numbers with knowledge of how to carry the result of the most significant bit. Also, just to put this criticism to rest, I redid the addition example using space-separated bits and verified the tokenization was one token per bit (but again, it shouldn't matter). Let's add the two binary numbers provided, limiting the work on each bit to a single line: 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1 -------------------------------------------------------------- 1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1 The answer is still wrong. (It also didn't use a single-line per bit, but that's ok).

Workaccount2 3y ago
I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. Most humans can do it with a pen and paper once told the rules. But you are giving them tools (and purpose designed methods) to bridge the gaps in humans own tokenizer. Who knows what the equivalent of pen and paper would be for an LLM.
- rcme 3y ago
  > I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. The prompt was "Now add 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 and 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1. Limit work on each bit to a single line." So the model could use its own output to show its work. Again, the rules are really simple and a human could do this trivially. All I'm saying, is that, to me, it looks like ChatGPT isn't capable for understanding logical steps. There are many, many examples online of people creating complicated logical puzzles that ChatGPT "solves." My hypothesis is that ChatGPT isn't actually "solving" these and ChatGPT would fail on any follow up examples that had sufficient entropy. To me, this is a significant difference. You might create a simple model of a problem and be amazed that ChatGPT seems to work, only to find that it fails with more complicated problems encountered in a real-world environment. That doesn't mean ChatGPT is useless, but it would be limited to automation purposes (operating on known examples) and could never be a general thinking agent.
  - Workaccount2 3y ago
    The reason humans can do it is because you give them a pen a paper so they can do it one step at a time. You are asking chatGPT to do the whole thing in one step. Give it pairs one at a time and ask it to output the binary value and carry value. You'll see it can do it just as easily as a human doing it step by step with pen and paper.
    - rcme 3y ago
      As far as I know, GPT does the work one token at a time. It produces a single token at each iteration. So if you give it "abc" and want "def" as the result, you first give it "abc" and it returns "d", then you give it "abcd" and it returns "e", then you give it "abcde" and it returns "f", then you give it "abcdef" and it returns the stop token. So just by outputting its work, it can provide its own context to memoize what the answer for each bit is.

by rcme 3y ago

Workaccount2 3y ago

I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. Most humans can do it with a pen and paper once told the rules. But you are giving them tools (and purpose designed methods) to bridge the gaps in humans own tokenizer. Who knows what the equivalent of pen and paper would be for an LLM.

rcme 3y ago
> I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. The prompt was "Now add 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 and 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1. Limit work on each bit to a single line." So the model could use its own output to show its work. Again, the rules are really simple and a human could do this trivially. All I'm saying, is that, to me, it looks like ChatGPT isn't capable for understanding logical steps. There are many, many examples online of people creating complicated logical puzzles that ChatGPT "solves." My hypothesis is that ChatGPT isn't actually "solving" these and ChatGPT would fail on any follow up examples that had sufficient entropy. To me, this is a significant difference. You might create a simple model of a problem and be amazed that ChatGPT seems to work, only to find that it fails with more complicated problems encountered in a real-world environment. That doesn't mean ChatGPT is useless, but it would be limited to automation purposes (operating on known examples) and could never be a general thinking agent.
- Workaccount2 3y ago
  The reason humans can do it is because you give them a pen a paper so they can do it one step at a time. You are asking chatGPT to do the whole thing in one step. Give it pairs one at a time and ask it to output the binary value and carry value. You'll see it can do it just as easily as a human doing it step by step with pen and paper.
  - rcme 3y ago
    As far as I know, GPT does the work one token at a time. It produces a single token at each iteration. So if you give it "abc" and want "def" as the result, you first give it "abc" and it returns "d", then you give it "abcd" and it returns "e", then you give it "abcde" and it returns "f", then you give it "abcdef" and it returns the stop token. So just by outputting its work, it can provide its own context to memoize what the answer for each bit is.