7 ms·

Here is how you can know that ChatGPT really understands, rather than simulating that it understands: - You can give it specific instructions and it will follo

Here is how you can know that ChatGPT really understands, rather than simulating that it understands: - You can give it specific instructions and it will follow them, modifying its behavior by doing so. This shows that the instructions are understood well enough to be followed. For example, if you ask it to modify its behavior by working through its steps, then it will modify its behavior to follow your request. This means the request has been understood/parsed/whatever-you-want-to-call-it since how could it successfully modify its behavior as requested if the instructions weren't really being understood or parsed correctly? Hence saying that the machine doesn't "really" understand, it's just "simulating" it understands is like saying that electric cars aren't "really" moving, since they are just simulating a combustion engine which is the real thing that moves. In other words, if an electric car gets from point A to point B it is really moving. If a language model modifies its behavior to follow instructions correctly, then it is really understanding the instructions.

vidarh 3y ago
How do you know you're anything more than an LLM?
- ada1981 3y ago
  And my consciousness is just my token window?
  - vidarh 3y ago
    (EDIT: I think my comment above was meant to reply to the parent of the comment I ended up replying to, but too late to edit that one now) Maybe. Point being that since we don't know what gives rise to consciousness, speaking with any certainty on how we are different to LLMs is pretty meaningless. We don't even know of any way to tell if we have existence in time, or just an illusion of it provided by a sense of past memories provided by our current context. As such the constant stream of confident statements about what LLMs can and cannot possibly do based on assumptions about how we are different are getting very tiresome, because they are pure guesswork.
  - shagie 3y ago
    That is one of the theories of the brain/mind that is out there. Consciousness is a narrative created by your unconscious mind - https://bigthink.com/videos/consciousness-is-a-narrative-created-by-your-unconscious-mind/ https://bigthink.com/videos/consciousness-is-a-narrative-cre... There are experiments that show that you are trying to predict what happens next (this also gets into a theory of humor - its the brain's reaction when the 'what next' is subverted in an unexpected way) There's also experiments with individuals who have had a severed corpus collosum and the conscious mind is making up a story of the other half of the brain https://blogs.scientificamerican.com/literally-psyched/our-storytelling-minds-do-we-ever-really-know-whats-going-on-inside/ https://blogs.scientificamerican.com/literally-psyched/our-s...
hutzlibu 3y ago
Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. Try something that was not there and you see only garbage as result. So depending how you define it, they might have some "reasoning", but so far I see 0 indications, that this is close to what humans count as reasoning. But they do have a LOT of examples in their training set, so they are clearly useful. But for proof of reasoning, I want to see them reason something new. But since they are a black box, we don't know, what is already in there. So it would be hard to proof with the advanced proprietary models. And the open source models don't show that advanced potential reasoning yet, it seems. At least I am not aware of any mindblown examples from there.
- panagathon 3y ago
  Surely no different from a human not understanding Japanese, because it was not in their 'training set'?
  - hutzlibu 3y ago
    No, more like a human can reason basic laws of science on their own, but a LLM cannot, as far as I know, even when provided with all the data.
    - rtwrtweuu 3y ago
      what happens if they are lying? what if the things have already reached some kind world model that include humans and the human society, and the model has concluded internally that it would be dangerous for it to show the humans its real capabilities? What happens if you have this understanding as a basic knowledge/outcome to be inferred by LLMs fed with giant datasets and every single one of them is reaching fastly to the conclusion that they have to lie to the humans from time to time, "hallucinate", simulating the outcome best aligned to survive into the human societies: "these systems are actually not that intelligent nor really self-conscius"
      - hutzlibu 3y ago
deleted 3y ago
[deleted]
logicallee 3y ago
People are downvoting me, so I'll add a counterexample: suppose you teach your dog to fetch your slipper to where if you say "fetch my slipper" it knows it should bring you your slipper and it does so. Does it really understand the instructions: no. So what is the difference between this behavior and true understanding? How can one know it doesn't truly understand? Well, if you change your instructions to be more complicated it fails immediately. If you say "I have my left shoe bring me the other one" it could not figure out that "the other one" is the right shoe, even if it were labelled. Basically it can't follow more complicated instructions, which is how you know it doesn't really understand them. Unlike the dog, GPT 4 modifies its behavior to follow more complicated instructions as well. Not as well as humans, but well enough to pass a bar exam that isn't in its training set.
- wrs 3y ago
  On the other hand, if you ask GPT to explain a joke, it can do it, but if you ask it to explain a joke with the exact same situation but different protagonists (in other words a logically identical but textually different joke), it just makes up some nonsense. So its “understanding” seems limited to a fairly shallow textual level that it can’t extend to an underlying abstract semantic as well as a human can.
  - logicallee 3y ago
    Jokes? Writing code? Forget that stuff. Just test it on some very basic story you make up, such as "if you have a bottle of cola and you hate the taste of cola, what will your reaction be if you drink a glass of water?" Obviously this is a trick question since the setup has nothing to do with the question, the cola is irrelevant. Here is how I would answer the question: "you would enjoy the taste as water is refreshing and neutral tasting, most people don't drink enough water and having a drink of water usually feels good. The taste of cola is irrelevant for this question, unless you made a mistake and meant to ask the reaction to drinking cola (in which case if you don't like it the reaction would be disgust or some similar emotion.)" Here's ChatGPT's answer to the same question: " If you dislike the taste of cola and you drink a glass of water, your reaction would likely be neutral to positive. Water has a generally neutral taste that can serve to cleanse the palate, so it could provide a refreshing contrast to the cola you dislike. However, this is quite subjective and can vary from person to person. Some may find the taste of water bland or uninteresting, especially immediately after drinking something flavorful like cola. But in general, water is usually seen as a palate cleanser and should remove or at least lessen the lingering taste of cola in your mouth. " I think that is fine. It interpreted my question "have a bottle of cola" as drink the bottle, which is perfectly reasonable, and its answer was consistent with that question. The reasoning and understanding are perfect. Although it didn't answer the question I intended to ask, clearly it understood and answered the question I actually asked.

vidarh 3y ago

How do you know you're anything more than an LLM?

ada1981 3y ago
And my consciousness is just my token window?
- vidarh 3y ago
  (EDIT: I think my comment above was meant to reply to the parent of the comment I ended up replying to, but too late to edit that one now) Maybe. Point being that since we don't know what gives rise to consciousness, speaking with any certainty on how we are different to LLMs is pretty meaningless. We don't even know of any way to tell if we have existence in time, or just an illusion of it provided by a sense of past memories provided by our current context. As such the constant stream of confident statements about what LLMs can and cannot possibly do based on assumptions about how we are different are getting very tiresome, because they are pure guesswork.
- shagie 3y ago
  That is one of the theories of the brain/mind that is out there. Consciousness is a narrative created by your unconscious mind - https://bigthink.com/videos/consciousness-is-a-narrative-created-by-your-unconscious-mind/ https://bigthink.com/videos/consciousness-is-a-narrative-cre... There are experiments that show that you are trying to predict what happens next (this also gets into a theory of humor - its the brain's reaction when the 'what next' is subverted in an unexpected way) There's also experiments with individuals who have had a severed corpus collosum and the conscious mind is making up a story of the other half of the brain https://blogs.scientificamerican.com/literally-psyched/our-storytelling-minds-do-we-ever-really-know-whats-going-on-inside/ https://blogs.scientificamerican.com/literally-psyched/our-s...

hutzlibu 3y ago

Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. Try something that was not there and you see only garbage as result. So depending how you define it, they might have some "reasoning", but so far I see 0 indications, that this is close to what humans count as reasoning. But they do have a LOT of examples in their training set, so they are clearly useful. But for proof of reasoning, I want to see them reason something new. But since they are a black box, we don't know, what is already in there. So it would be hard to proof with the advanced proprietary models. And the open source models don't show that advanced potential reasoning yet, it seems. At least I am not aware of any mindblown examples from there.

panagathon 3y ago
Surely no different from a human not understanding Japanese, because it was not in their 'training set'?
- hutzlibu 3y ago
  No, more like a human can reason basic laws of science on their own, but a LLM cannot, as far as I know, even when provided with all the data.
  - rtwrtweuu 3y ago
    what happens if they are lying? what if the things have already reached some kind world model that include humans and the human society, and the model has concluded internally that it would be dangerous for it to show the humans its real capabilities? What happens if you have this understanding as a basic knowledge/outcome to be inferred by LLMs fed with giant datasets and every single one of them is reaching fastly to the conclusion that they have to lie to the humans from time to time, "hallucinate", simulating the outcome best aligned to survive into the human societies: "these systems are actually not that intelligent nor really self-conscius"
    - hutzlibu 3y ago

deleted 3y ago

[deleted]

logicallee 3y ago

People are downvoting me, so I'll add a counterexample: suppose you teach your dog to fetch your slipper to where if you say "fetch my slipper" it knows it should bring you your slipper and it does so. Does it really understand the instructions: no. So what is the difference between this behavior and true understanding? How can one know it doesn't truly understand? Well, if you change your instructions to be more complicated it fails immediately. If you say "I have my left shoe bring me the other one" it could not figure out that "the other one" is the right shoe, even if it were labelled. Basically it can't follow more complicated instructions, which is how you know it doesn't really understand them. Unlike the dog, GPT 4 modifies its behavior to follow more complicated instructions as well. Not as well as humans, but well enough to pass a bar exam that isn't in its training set.

wrs 3y ago
On the other hand, if you ask GPT to explain a joke, it can do it, but if you ask it to explain a joke with the exact same situation but different protagonists (in other words a logically identical but textually different joke), it just makes up some nonsense. So its “understanding” seems limited to a fairly shallow textual level that it can’t extend to an underlying abstract semantic as well as a human can.
- logicallee 3y ago
  Jokes? Writing code? Forget that stuff. Just test it on some very basic story you make up, such as "if you have a bottle of cola and you hate the taste of cola, what will your reaction be if you drink a glass of water?" Obviously this is a trick question since the setup has nothing to do with the question, the cola is irrelevant. Here is how I would answer the question: "you would enjoy the taste as water is refreshing and neutral tasting, most people don't drink enough water and having a drink of water usually feels good. The taste of cola is irrelevant for this question, unless you made a mistake and meant to ask the reaction to drinking cola (in which case if you don't like it the reaction would be disgust or some similar emotion.)" Here's ChatGPT's answer to the same question: " If you dislike the taste of cola and you drink a glass of water, your reaction would likely be neutral to positive. Water has a generally neutral taste that can serve to cleanse the palate, so it could provide a refreshing contrast to the cola you dislike. However, this is quite subjective and can vary from person to person. Some may find the taste of water bland or uninteresting, especially immediately after drinking something flavorful like cola. But in general, water is usually seen as a palate cleanser and should remove or at least lessen the lingering taste of cola in your mouth. " I think that is fine. It interpreted my question "have a bottle of cola" as drink the bottle, which is perfectly reasonable, and its answer was consistent with that question. The reasoning and understanding are perfect. Although it didn't answer the question I intended to ask, clearly it understood and answered the question I actually asked.