4 ms·

>The other day I saw this Twitter thread. Briefly: GPT knows many human languages, InstructGPT is GPT plus some finetuning in English. Then they fed InstructGPT

by foldr 4y ago

>The other day I saw this Twitter thread. Briefly: GPT knows many human languages, InstructGPT is GPT plus some finetuning in English. Then they fed InstructGPT requests in some other human language, and it carries them out, following the English-language finetuning. >And I thought: so what? Isn’t this expected behaviour? Then a friend pointed out that this is only confusing if you think InstructGPT doesn’t understand concepts. > [conclusion that ChatGPT must understand concepts] I think this argument is a bit mixed up. Good quality machine translation has been possible for longer than ChatGPT has been around. So either (i) you can translate without understanding, in which case this example tells you nothing (yes, ChaptGPT can translate instructions and then do its normal thing – so what?), or (ii) you can't translate without understanding, in which case you could just use machine translation as your example to show that some computational model is capable of understanding, and leave ChatGPT out of it.

deleted 4y ago
[deleted]
_dain_ 4y ago
The point isn't that it can translate between languages. It's not translating the instructions, at least not explicitly. Here's what they did: - They found a task that GPT wasn't very good at, because examples of that task weren't in the training set (in any language). - They trained a fine-tuned variant of GPT where examples of the task were in an appended training set, but only in English. - They told the variant to do that task again, in other languages. - Its performance improved on all of them, not just English If the training set has no Mandarin examples of the task, how did it get better when you ask it in Mandarin? Sure, you could fake this by having an API call to Google Translate where it takes the Mandarin request, translates to English, solves the task in English, then Google Translates back to Mandarin. But it's not doing that, it's just doing the same "predict the next token" operation on the Mandarin prompt. I don't see how it can do that unless it really has some kind of understanding.
- ivanbakel 4y ago
  >I don't see how it can do that unless it really has some kind of understanding. One possibility is that the model itself has learned that tokens are related across languages based on translation examples. If the appended training changes the model's treatment of tokens in one language, that could have a statistical knock-on effect on the weights between similar tokens in different languages. Similarly, if you train the model that "blue" is a "colour", you'd expect it to pick up that "navy" is a "shade".
  - _dain_ 4y ago
    At the scale/complexity GPT operates at, how is that different from "understanding"? It seems like you've just rephrased it but with more words.
    - majewsky 4y ago
      Yeah, but then it goes back to GP's original argument. If relations between translated tokens are classified as "understanding", that would mean that translation AIs are already capable of understanding: > in which case you could just use machine translation as your example to show that some computational model is capable of understanding, and leave ChatGPT out of it.
      - _dain_ 4y ago
        Show me a pre-GPT machine translation model that can do what I've described.
        deleted 4y ago
        [deleted]
    - ivanbakel 4y ago
      Because it's a purely statistical explanation that doesn't require understanding. Put differently, it's possible that GPT doesn't "understand" language itself as a concept, and instead tokens in the same language are just highly-correlated when it comes to prediction. When affecting weights between tokens, it wouldn't be surprising that those weights have effects across languages, much in the same way they work within languages - after all, it's all just probabilities to GPT. Google Translate works in a comparable way, and nobody suggests that it is sentient. Frankly, the argument that "GPT operates at a scale that means it must be sentient" is begging the question.

by foldr 4y ago

deleted 4y ago

[deleted]

_dain_ 4y ago

The point isn't that it can translate between languages. It's not translating the instructions, at least not explicitly. Here's what they did: - They found a task that GPT wasn't very good at, because examples of that task weren't in the training set (in any language). - They trained a fine-tuned variant of GPT where examples of the task were in an appended training set, but only in English. - They told the variant to do that task again, in other languages. - Its performance improved on all of them, not just English If the training set has no Mandarin examples of the task, how did it get better when you ask it in Mandarin? Sure, you could fake this by having an API call to Google Translate where it takes the Mandarin request, translates to English, solves the task in English, then Google Translates back to Mandarin. But it's not doing that, it's just doing the same "predict the next token" operation on the Mandarin prompt. I don't see how it can do that unless it really has some kind of understanding.

ivanbakel 4y ago
>I don't see how it can do that unless it really has some kind of understanding. One possibility is that the model itself has learned that tokens are related across languages based on translation examples. If the appended training changes the model's treatment of tokens in one language, that could have a statistical knock-on effect on the weights between similar tokens in different languages. Similarly, if you train the model that "blue" is a "colour", you'd expect it to pick up that "navy" is a "shade".
- _dain_ 4y ago
  At the scale/complexity GPT operates at, how is that different from "understanding"? It seems like you've just rephrased it but with more words.
  - majewsky 4y ago
    Yeah, but then it goes back to GP's original argument. If relations between translated tokens are classified as "understanding", that would mean that translation AIs are already capable of understanding: > in which case you could just use machine translation as your example to show that some computational model is capable of understanding, and leave ChatGPT out of it.
    - _dain_ 4y ago
      Show me a pre-GPT machine translation model that can do what I've described.
      - deleted 4y ago
        [deleted]
  - ivanbakel 4y ago
    Because it's a purely statistical explanation that doesn't require understanding. Put differently, it's possible that GPT doesn't "understand" language itself as a concept, and instead tokens in the same language are just highly-correlated when it comes to prediction. When affecting weights between tokens, it wouldn't be surprising that those weights have effects across languages, much in the same way they work within languages - after all, it's all just probabilities to GPT. Google Translate works in a comparable way, and nobody suggests that it is sentient. Frankly, the argument that "GPT operates at a scale that means it must be sentient" is begging the question.