4 ms·

There's a lot of answers here, but as Feynman said "that which I can't build I don't understand." If you can't make something that writes for you, you don't rea

There's a lot of answers here, but as Feynman said "that which I can't build I don't understand." If you can't make something that writes for you, you don't really understand how you write. That feels impossible, to be able to do something without understanding how you do it. Brain be like that sometimes.

leereeves 4y ago
Understanding how ChatGPT writes doesn't translate to understanding how humans write/speak/think. We still don't understand the latter.
- IIAOPSW 4y ago
  Notice I didn't say anything about chatGPT. chatGPT is at least something that's been built. Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Let's call it a "potential understanding". This is different from an "illusory understanding", which feel like explanations but when you go to implement it you realize it was all a word game and your lack of understanding has just been swept under different words or under equally hard to understand concepts. An illusory understanding doesn't even have the potential to be an understanding. You can't use it to build your own version of GPTchat. All these glib answers people here are giving aren't even potential understandings. They are illusory understandings. They sound nice so long as you don't try to use it as a spec to code up your own language model. If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. You have a feeling of understanding.
  - leereeves 4y ago
    > Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Yeah, and it helps us understand how "overgrown text prediction" works, but not how humans work. In the same way that building a robotic arm won't help you understand how muscles work. > If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. Not entirely true. We can't build a star, but we do have a pretty good theory of how stars work. But I do agree that we don't understand the human brain.
    - IIAOPSW 4y ago
      Building a robot arm does help understand how muscles work. By having a robot arm to play with, you can pinpoint exactly the abilities which aren't accounted for by your current understanding. Boston Dynamics built that robot dog thing. It took decades to make robotic kinematics that good. The dog demonstrates an understanding of a system of muscles that solves the same problems as its biological counterpart. You say that overgrown text prediction isn't how humans work. And full disclosure you're probably right. But me put on my contrarian hat and say that actually you're wrong and that really is all there is to the brain. At what point does my theory break? What types of things can't be done with just overgrown text prediction, and what features are relevant to a system that could do those things? Don't just appeal to intuition and tell me humans obviously don't work that way. Find the actual flaw where the theory breaks down. That is the value of this experiment. If you can find the words / experiments to demonstrate why overgrown text prediction isn't an accurate understanding of human thought, in the process you will have in fact distilled a better understanding of human thought. Information on how the brain doesn't work is also information about how the brain works.
      - leereeves 4y ago
        > Building a robot arm does help understand how muscles work. I think we're talking about different levels. A robotic arm helps understand the mechanics of an arm, but not cell metabolism, myosin motors, etc. Any understanding of muscles you might get from a robotic arm is superficial. > You say that overgrown text prediction isn't how humans work. To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers. We know that humans have some pure text prediction ability. People who've seen Mary Poppins can complete supercalifragili... even though it has no meaning. But how? We don't know, even after building LMs. > What types of things can't be done with just overgrown text prediction That's a different claim and not one I'm making. What types of things do humans not do with text prediction? Anything that doesn't involve the language processing parts of the brain, at least.
        4y ago
JellyBeanThief 4y ago
I'd be careful about interpreting that. Can you "build" numbers? Yes, in one sense. No in another. And he doesn't specify whether being able to build something is sufficient for understanding it or merely necessary. And even when you've built it, what about other ways that it could be built? If you implement binary search iteratively, then perhaps you understand binary search. But do you understand its recursive implementation?
- pixl97 4y ago
  In addition, humanity has 'built' compounds for millenia without a deep understanding of chemistry and the atomic model. I would perhaps say "I cannot perfect that I do not understand", but noone is writing my musings in books.

leereeves 4y ago

Understanding how ChatGPT writes doesn't translate to understanding how humans write/speak/think. We still don't understand the latter.

IIAOPSW 4y ago
Notice I didn't say anything about chatGPT. chatGPT is at least something that's been built. Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Let's call it a "potential understanding". This is different from an "illusory understanding", which feel like explanations but when you go to implement it you realize it was all a word game and your lack of understanding has just been swept under different words or under equally hard to understand concepts. An illusory understanding doesn't even have the potential to be an understanding. You can't use it to build your own version of GPTchat. All these glib answers people here are giving aren't even potential understandings. They are illusory understandings. They sound nice so long as you don't try to use it as a spec to code up your own language model. If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. You have a feeling of understanding.
- leereeves 4y ago
  > Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Yeah, and it helps us understand how "overgrown text prediction" works, but not how humans work. In the same way that building a robotic arm won't help you understand how muscles work. > If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. Not entirely true. We can't build a star, but we do have a pretty good theory of how stars work. But I do agree that we don't understand the human brain.
  - IIAOPSW 4y ago
    Building a robot arm does help understand how muscles work. By having a robot arm to play with, you can pinpoint exactly the abilities which aren't accounted for by your current understanding. Boston Dynamics built that robot dog thing. It took decades to make robotic kinematics that good. The dog demonstrates an understanding of a system of muscles that solves the same problems as its biological counterpart. You say that overgrown text prediction isn't how humans work. And full disclosure you're probably right. But me put on my contrarian hat and say that actually you're wrong and that really is all there is to the brain. At what point does my theory break? What types of things can't be done with just overgrown text prediction, and what features are relevant to a system that could do those things? Don't just appeal to intuition and tell me humans obviously don't work that way. Find the actual flaw where the theory breaks down. That is the value of this experiment. If you can find the words / experiments to demonstrate why overgrown text prediction isn't an accurate understanding of human thought, in the process you will have in fact distilled a better understanding of human thought. Information on how the brain doesn't work is also information about how the brain works.
    - leereeves 4y ago
      > Building a robot arm does help understand how muscles work. I think we're talking about different levels. A robotic arm helps understand the mechanics of an arm, but not cell metabolism, myosin motors, etc. Any understanding of muscles you might get from a robotic arm is superficial. > You say that overgrown text prediction isn't how humans work. To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers. We know that humans have some pure text prediction ability. People who've seen Mary Poppins can complete supercalifragili... even though it has no meaning. But how? We don't know, even after building LMs. > What types of things can't be done with just overgrown text prediction That's a different claim and not one I'm making. What types of things do humans not do with text prediction? Anything that doesn't involve the language processing parts of the brain, at least.
      - 4y ago

JellyBeanThief 4y ago

I'd be careful about interpreting that. Can you "build" numbers? Yes, in one sense. No in another. And he doesn't specify whether being able to build something is sufficient for understanding it or merely necessary. And even when you've built it, what about other ways that it could be built? If you implement binary search iteratively, then perhaps you understand binary search. But do you understand its recursive implementation?

pixl97 4y ago
In addition, humanity has 'built' compounds for millenia without a deep understanding of chemistry and the atomic model. I would perhaps say "I cannot perfect that I do not understand", but noone is writing my musings in books.