3 ms·

I don't understand how you can write a whole piece on Transformers failing to do arithmetics without mentioning the tokenizer used in GPT, which makes it more o

by sva_ 4y ago

I don't understand how you can write a whole piece on Transformers failing to do arithmetics without mentioning the tokenizer used in GPT, which makes it more or less impossible for the model. The reason is imo probably more along the lines of that the model can't see the actual numbers you're giving it. When you provide GPT3 with the number "10011111110100011100011011110010", it actually sees [3064, 26259, 1157, 8784, 18005, 1157, 18005, 8784, 1157, 3064, 940] [see 0]. The number isn't even split up into even parts. Even if it knew the Peano axioms, it wouldn't have a chance because it can't 'see' the actual numbers that you're feeding it. Hence it can only estimate these operations based on combination pairs it has seen in training data, and it seems pretty obvious why this can't be a robust basis for doing arithmetic. 0. https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer

valine 4y ago
It also completely ignores the developments around GPT tool use. If you’re evaluating the AI field as a whole, why are you restricting your analysis to the raw output of GPTs? Arguably the most impressive display of intelligence exhibited by these models is their ability to use tools like calculators to augment their own abilities. It doesn’t matter if the model can’t do three digit multiplication without access to a calculator. Most people can’t mentally multiply large numbers either.
- rcme 4y ago
  Most people can do 32-bit binary addition using a pen and paper. And the point of using addition as an illustrative example is that the rules of addition are extremely simple, yet a binary addition problem can easily be scaled to N bits of entropy. This means it's very easy to show that the system fails to apply simple logic rules in such cases.
  - carrolldunham 4y ago
    It didn't fail logic, it failed to read your string. You didn't even notice that the terms being added in the 'show working' version don't match the given strings. And you are elsewhere talking about adding spaces to make the digits 'tokens'... you have no idea what you are talking about at all.
    - rcme 4y ago
      You can input the space-separated strings into OpenAI's tokenizer: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer You'll see that it does properly tokenize each binary digital as a separate token. But again, the tokenization doesn't matter. I could say to you, please add these two binary numbers: 101 0011 11001 + 00 11 10 100011 And you could do it.
  - valine 4y ago
    If your goal is to simulate a human doing addition with pen and paper it only seems fair to give the GPT read/write access to a text file. Outputting the answer directly from the model is much more akin to mental math.
    - rcme 4y ago
      ChatGPT does have read write access to a text file. It's allowed to show its work as part of its output, which it can write tokens to, and then read those tokens back as part of generating the next tokens.
rcme 4y ago
Why does the tokenization matter? Also, ChatGPT doesn't "see" those numbers, it just sees smaller binary strings, and a human could add that series of binary numbers with knowledge of how to carry the result of the most significant bit. Also, just to put this criticism to rest, I redid the addition example using space-separated bits and verified the tokenization was one token per bit (but again, it shouldn't matter). Let's add the two binary numbers provided, limiting the work on each bit to a single line: 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1 -------------------------------------------------------------- 1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1 The answer is still wrong. (It also didn't use a single-line per bit, but that's ok).
- Workaccount2 4y ago
  I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. Most humans can do it with a pen and paper once told the rules. But you are giving them tools (and purpose designed methods) to bridge the gaps in humans own tokenizer. Who knows what the equivalent of pen and paper would be for an LLM.
  - rcme 4y ago
    > I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. The prompt was "Now add 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 and 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1. Limit work on each bit to a single line." So the model could use its own output to show its work. Again, the rules are really simple and a human could do this trivially. All I'm saying, is that, to me, it looks like ChatGPT isn't capable for understanding logical steps. There are many, many examples online of people creating complicated logical puzzles that ChatGPT "solves." My hypothesis is that ChatGPT isn't actually "solving" these and ChatGPT would fail on any follow up examples that had sufficient entropy. To me, this is a significant difference. You might create a simple model of a problem and be amazed that ChatGPT seems to work, only to find that it fails with more complicated problems encountered in a real-world environment. That doesn't mean ChatGPT is useless, but it would be limited to automation purposes (operating on known examples) and could never be a general thinking agent.

by sva_ 4y ago

valine 4y ago

It also completely ignores the developments around GPT tool use. If you’re evaluating the AI field as a whole, why are you restricting your analysis to the raw output of GPTs? Arguably the most impressive display of intelligence exhibited by these models is their ability to use tools like calculators to augment their own abilities. It doesn’t matter if the model can’t do three digit multiplication without access to a calculator. Most people can’t mentally multiply large numbers either.

rcme 4y ago
Most people can do 32-bit binary addition using a pen and paper. And the point of using addition as an illustrative example is that the rules of addition are extremely simple, yet a binary addition problem can easily be scaled to N bits of entropy. This means it's very easy to show that the system fails to apply simple logic rules in such cases.
- carrolldunham 4y ago
  It didn't fail logic, it failed to read your string. You didn't even notice that the terms being added in the 'show working' version don't match the given strings. And you are elsewhere talking about adding spaces to make the digits 'tokens'... you have no idea what you are talking about at all.
  - rcme 4y ago
    You can input the space-separated strings into OpenAI's tokenizer: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer You'll see that it does properly tokenize each binary digital as a separate token. But again, the tokenization doesn't matter. I could say to you, please add these two binary numbers: 101 0011 11001 + 00 11 10 100011 And you could do it.
- valine 4y ago
  If your goal is to simulate a human doing addition with pen and paper it only seems fair to give the GPT read/write access to a text file. Outputting the answer directly from the model is much more akin to mental math.
  - rcme 4y ago
    ChatGPT does have read write access to a text file. It's allowed to show its work as part of its output, which it can write tokens to, and then read those tokens back as part of generating the next tokens.

rcme 4y ago

Why does the tokenization matter? Also, ChatGPT doesn't "see" those numbers, it just sees smaller binary strings, and a human could add that series of binary numbers with knowledge of how to carry the result of the most significant bit. Also, just to put this criticism to rest, I redid the addition example using space-separated bits and verified the tokenization was one token per bit (but again, it shouldn't matter). Let's add the two binary numbers provided, limiting the work on each bit to a single line: 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1 -------------------------------------------------------------- 1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1 The answer is still wrong. (It also didn't use a single-line per bit, but that's ok).

Workaccount2 4y ago
I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. Most humans can do it with a pen and paper once told the rules. But you are giving them tools (and purpose designed methods) to bridge the gaps in humans own tokenizer. Who knows what the equivalent of pen and paper would be for an LLM.
- rcme 4y ago
  > I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too. The prompt was "Now add 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 and 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1. Limit work on each bit to a single line." So the model could use its own output to show its work. Again, the rules are really simple and a human could do this trivially. All I'm saying, is that, to me, it looks like ChatGPT isn't capable for understanding logical steps. There are many, many examples online of people creating complicated logical puzzles that ChatGPT "solves." My hypothesis is that ChatGPT isn't actually "solving" these and ChatGPT would fail on any follow up examples that had sufficient entropy. To me, this is a significant difference. You might create a simple model of a problem and be amazed that ChatGPT seems to work, only to find that it fails with more complicated problems encountered in a real-world environment. That doesn't mean ChatGPT is useless, but it would be limited to automation purposes (operating on known examples) and could never be a general thinking agent.