3 ms·

What a disappointing article. So many reasonable explanations of technically correct descriptions of forward passes, backprop, gradient descent, and then at th

What a disappointing article. So many reasonable explanations of technically correct descriptions of forward passes, backprop, gradient descent, and then at the end: > but ChatGPT has no actual conception of what it’s talking about—no internal model of a stuck sandwich on which it can experiment with different strategies for removal. You can both see for yourself that GPT-4 has a (somewhat unreliable) world model by just using it with mildly creative questioning, and you can also explain it with the technical knowledge that the article's author appears to already have: > The A.I. is simply remixing and recombining existing writing that’s relevant to the prompt. Look, at this point there are entire, well known, 150 page papers pointing out that it can do things like successfully predict what some source code you feed it -- even if it is pseudo-code -- would output if it were executed. The amount of load-bearing cognitive dissonance necessary to see that and then say that it only knows how to shuffle words it's already seen without reasoning about anything must be immense.

mdp2021 4y ago
> successfully predict what some source code you feed it ... would output if it were executed Ok, but then the important question is "how". Your job, with the aim of using fire, is to know fire in order to control fire.
- cjbprime 4y ago
  How is it completing the task? The only possible way to perform that task reliably is to have a principled understanding of what the code actually does, and to create a world model based on that understanding, and then extract consequences from that world model. You can reassure yourself that GPT-4 is actually doing this by asking it to share its modeling of the code with you. How is it able to do that? By being so grand in scale that it does not merely model the likelihood of a token based on the text it's seen before, but on the causes of that text having been written in the first place. It was a fallacy to expect that it would not do this if the parameters and dataset were sufficiently large. It's a fallacy that is still happening, including in popular New Yorker articles.
  - strange_quark 4y ago
    It’s also read probably countless CS textbooks and tutorials, all of StackOverflow, all of GitHub, etc. I think it’s really significantly less of a stretch that to state that it can do what you’re describing because a lot of code is similar and it’s seen gigabytes of it, rather than state it somehow learned how to reason in the human sense.
    - cjbprime 4y ago
      To be clear, this is not "what does this line of code do?", but "what does this many-line program do?". Even if you've seen all the lines individually before, you still have to work out what happens when you combine them together. If most of the combination was not in the training set, that's reasoning to me.
  - mdp2021 4y ago
    But it remains that we want to see how this works precisely enough to make it work reliably. (And "possibly" to learn from that - because, you know, otherwise we have a tool but no knowledge. Also consistently with the example of the fire.)

mdp2021 4y ago

> successfully predict what some source code you feed it ... would output if it were executed Ok, but then the important question is "how". Your job, with the aim of using fire, is to know fire in order to control fire.

cjbprime 4y ago
How is it completing the task? The only possible way to perform that task reliably is to have a principled understanding of what the code actually does, and to create a world model based on that understanding, and then extract consequences from that world model. You can reassure yourself that GPT-4 is actually doing this by asking it to share its modeling of the code with you. How is it able to do that? By being so grand in scale that it does not merely model the likelihood of a token based on the text it's seen before, but on the causes of that text having been written in the first place. It was a fallacy to expect that it would not do this if the parameters and dataset were sufficiently large. It's a fallacy that is still happening, including in popular New Yorker articles.
- strange_quark 4y ago
  It’s also read probably countless CS textbooks and tutorials, all of StackOverflow, all of GitHub, etc. I think it’s really significantly less of a stretch that to state that it can do what you’re describing because a lot of code is similar and it’s seen gigabytes of it, rather than state it somehow learned how to reason in the human sense.
  - cjbprime 4y ago
    To be clear, this is not "what does this line of code do?", but "what does this many-line program do?". Even if you've seen all the lines individually before, you still have to work out what happens when you combine them together. If most of the combination was not in the training set, that's reasoning to me.
- mdp2021 4y ago
  But it remains that we want to see how this works precisely enough to make it work reliably. (And "possibly" to learn from that - because, you know, otherwise we have a tool but no knowledge. Also consistently with the example of the fire.)