8 ms·

I still hold that it doesn't "understand". Even if it answered all questions perfectly, stopped making mistakes, and produced fully working programs better than

by _xnmw 4y ago

I still hold that it doesn't "understand". Even if it answered all questions perfectly, stopped making mistakes, and produced fully working programs better than the best crack developer teams, that still doesn't mean it "understands". "Understanding" is not an output, it's a process, that is sometimes (but not always) measured by its output.

_dain_ 4y ago
By that standard, how do you know another human being understands anything? All you see is their behaviour. You don't have access to their internals, you don't really know what "process" is going on in there. This road leads to solipsism.
famouswaffles 4y ago
This is nonsensical lol. But watching the posts shift in real time is very entertaining.
marcosdumay 4y ago
If it stopped making mistakes and produced complete fully working programs, no, there would be no way to say it doesn't understand. Yes, "understanding" is a process, but it's not well defined. And anyway, if it's a requirement for those things, and the AI did those things, then the only possibility is that the AI has this process in some way. But well, our current AIs do not produce complete programs, nor fully working ones, nor do they say things without making mistakes. All the people making assumptions about the next generation that will do those things are basically hyping bullshit; and the next generation won't do those things because those AIs don't understand. What doesn't mean that eventually an AI that understands won't appear; of course it will. It just won't be the next generation of those.
- _dain_ 4y ago
  What makes you so confident? Did you accurately predict in 2021 what the SOTA LLM capabilities would be in 2023?
- YeGoblynQueenne 4y ago
  >> If it stopped making mistakes and produced complete fully working programs, no, there would be no way to say it doesn't understand. There exist program synthesis systems that always produce "complete fully working programs" and that in fact cannot make any mistakes because they are based on algorithms with strong theoretical guarantees (like mathematical proofs) of their correctness. You give them a set of examples and some "background knowledge" and they spit out a program that's consistent with the examples and the background knowledge, no ifs and buts and maybes. I could point you to several systems like that, if you wish, but first I want to make sure that we both agree what you are saying: a program synthesis system that never makes mistakes "understands". Is that right? Could you also please clarify what such a system "understands"? Then I'm happy to link you to some systems like the ones I'm talking about.
  - marcosdumay 4y ago
    > a program synthesis system that never makes mistakes "understands" Yes, I'm saying if it satisfies random requirements and synthesizes full working programs, then it has some form of understanding in it. If creates that understanding through learning, it's very possibly a general form of AI (I'm not certain of this part). I've never seen any one that arrived at structure by learning. It's always programed in, what puts the intelligence on a very conventional place, and I doubt you'll disagree that it understands what it's doing.
    - YeGoblynQueenne 4y ago
      Didn't you just shift the goal posts? You care about how the structure is arrived at? Anyway the systems I have in mind do learn their own structure, I know at least three that do that. But you're not going to get machine learning without predetermined structure. In Transformers, as in all neural nets, the structure is right there in the architecture of the neural net. So what kind of "learning" without "structure" exactly are you looking for?
      - marcosdumay 4y ago
        > Didn't you just shift the goal posts? Nope, you changed the context from a zero-knowledge AI optimized for text into things that come with knowledge designed in. There is understanding embedded on the things with knowledge designed in, but the understanding is not done by the AI. Anyway, I'm surely curious about the systems that learn their own structure. You mean they learn the syntax rules? I'd say they have some understanding of the language syntax, but well, unless they can take a requirement and produce working code, they don't understand programing. (And well, if the requirement is in a formal language, they only understand formal transformations, that is, your AI understands compiling. What honestly, is quite limited, but would still be impressive.)
- _xnmw 4y ago
  Quality of output does not mean that the process is genuine. A well-produced movie with good actors may depict a war better than footage of an actual war, but that is not evidence of an actual war happening. Statistical LLMs are trying really hard at "acting" to produce output that looks like there is genuine understanding, but there is no understanding going on, regardless of how good the output looks.
wging 4y ago
If that’s true, then you’ll need to be careful to decouple the phrase “it doesn’t understand” from your predictions about its behavior. I think most people who say it doesn’t understand intend to imply a deficiency of capability, not just an internal state.
pornel 4y ago
To me setting the bar so high makes that definition of "understanding" meaningless. If AI's outputs were literally indistinguishable from something that truly understands these concepts, then there's no practical difference if it really "understands". This reduces the concept of understanding to some abstract inconsequential property.
- _xnmw 4y ago
  > If AI's outputs were literally indistinguishable from something that truly understands these concepts, then there's no practical difference if it really "understands". This reduces the concept of understanding to some abstract inconsequential property. Yes, there is a practical difference. It has consequences in terms of legality, blame, and justice, the other does not. If an AI really "understands" things, then it is responsible for its actions and must be taken to court for crimes, and can be sued for mistakes in the output that result in losses (like any lawyer or professional).
  - pornel 4y ago
    That is a huge leap into personhood and very human concepts that depend on having volition, motives, feelings, and self-awareness or consciousness. I don't think these are necessary connected to understanding. IMHO it's possible to have full understanding of all of the concepts in the world while only seeing the world as a data snapshot, without being an entity capable of participating in it, without having any feelings, self-awareness, or even any sensory inputs or memory of own experiences. Laws, courts, and their punishments are for humans, and they're likely to be absurdly bad at dealing with a personified AI. AI won't feel fear or remorse, may not have self-preservation as a goal, so it may not care about any threats at all. It may not even have sensory input needed to be aware of being punished.
perryizgr8 4y ago
You can say this same thing about any human too. After all, how can you be sure that my mind "understands" the same way your does? If your argument is that we have similar hardware, you must point out what's so special about this biological brain that enables "understanding" and why neural networks don't.

by _xnmw 4y ago

_dain_ 4y ago

By that standard, how do you know another human being understands anything? All you see is their behaviour. You don't have access to their internals, you don't really know what "process" is going on in there. This road leads to solipsism.

famouswaffles 4y ago

This is nonsensical lol. But watching the posts shift in real time is very entertaining.

marcosdumay 4y ago

If it stopped making mistakes and produced complete fully working programs, no, there would be no way to say it doesn't understand. Yes, "understanding" is a process, but it's not well defined. And anyway, if it's a requirement for those things, and the AI did those things, then the only possibility is that the AI has this process in some way. But well, our current AIs do not produce complete programs, nor fully working ones, nor do they say things without making mistakes. All the people making assumptions about the next generation that will do those things are basically hyping bullshit; and the next generation won't do those things because those AIs don't understand. What doesn't mean that eventually an AI that understands won't appear; of course it will. It just won't be the next generation of those.

_dain_ 4y ago
What makes you so confident? Did you accurately predict in 2021 what the SOTA LLM capabilities would be in 2023?
YeGoblynQueenne 4y ago
>> If it stopped making mistakes and produced complete fully working programs, no, there would be no way to say it doesn't understand. There exist program synthesis systems that always produce "complete fully working programs" and that in fact cannot make any mistakes because they are based on algorithms with strong theoretical guarantees (like mathematical proofs) of their correctness. You give them a set of examples and some "background knowledge" and they spit out a program that's consistent with the examples and the background knowledge, no ifs and buts and maybes. I could point you to several systems like that, if you wish, but first I want to make sure that we both agree what you are saying: a program synthesis system that never makes mistakes "understands". Is that right? Could you also please clarify what such a system "understands"? Then I'm happy to link you to some systems like the ones I'm talking about.
- marcosdumay 4y ago
  > a program synthesis system that never makes mistakes "understands" Yes, I'm saying if it satisfies random requirements and synthesizes full working programs, then it has some form of understanding in it. If creates that understanding through learning, it's very possibly a general form of AI (I'm not certain of this part). I've never seen any one that arrived at structure by learning. It's always programed in, what puts the intelligence on a very conventional place, and I doubt you'll disagree that it understands what it's doing.
  - YeGoblynQueenne 4y ago
    Didn't you just shift the goal posts? You care about how the structure is arrived at? Anyway the systems I have in mind do learn their own structure, I know at least three that do that. But you're not going to get machine learning without predetermined structure. In Transformers, as in all neural nets, the structure is right there in the architecture of the neural net. So what kind of "learning" without "structure" exactly are you looking for?
    - marcosdumay 4y ago
      > Didn't you just shift the goal posts? Nope, you changed the context from a zero-knowledge AI optimized for text into things that come with knowledge designed in. There is understanding embedded on the things with knowledge designed in, but the understanding is not done by the AI. Anyway, I'm surely curious about the systems that learn their own structure. You mean they learn the syntax rules? I'd say they have some understanding of the language syntax, but well, unless they can take a requirement and produce working code, they don't understand programing. (And well, if the requirement is in a formal language, they only understand formal transformations, that is, your AI understands compiling. What honestly, is quite limited, but would still be impressive.)
_xnmw 4y ago
Quality of output does not mean that the process is genuine. A well-produced movie with good actors may depict a war better than footage of an actual war, but that is not evidence of an actual war happening. Statistical LLMs are trying really hard at "acting" to produce output that looks like there is genuine understanding, but there is no understanding going on, regardless of how good the output looks.

wging 4y ago

If that’s true, then you’ll need to be careful to decouple the phrase “it doesn’t understand” from your predictions about its behavior. I think most people who say it doesn’t understand intend to imply a deficiency of capability, not just an internal state.

pornel 4y ago

To me setting the bar so high makes that definition of "understanding" meaningless. If AI's outputs were literally indistinguishable from something that truly understands these concepts, then there's no practical difference if it really "understands". This reduces the concept of understanding to some abstract inconsequential property.

_xnmw 4y ago
> If AI's outputs were literally indistinguishable from something that truly understands these concepts, then there's no practical difference if it really "understands". This reduces the concept of understanding to some abstract inconsequential property. Yes, there is a practical difference. It has consequences in terms of legality, blame, and justice, the other does not. If an AI really "understands" things, then it is responsible for its actions and must be taken to court for crimes, and can be sued for mistakes in the output that result in losses (like any lawyer or professional).
- pornel 4y ago
  That is a huge leap into personhood and very human concepts that depend on having volition, motives, feelings, and self-awareness or consciousness. I don't think these are necessary connected to understanding. IMHO it's possible to have full understanding of all of the concepts in the world while only seeing the world as a data snapshot, without being an entity capable of participating in it, without having any feelings, self-awareness, or even any sensory inputs or memory of own experiences. Laws, courts, and their punishments are for humans, and they're likely to be absurdly bad at dealing with a personified AI. AI won't feel fear or remorse, may not have self-preservation as a goal, so it may not care about any threats at all. It may not even have sensory input needed to be aware of being punished.

perryizgr8 4y ago

You can say this same thing about any human too. After all, how can you be sure that my mind "understands" the same way your does? If your argument is that we have similar hardware, you must point out what's so special about this biological brain that enables "understanding" and why neural networks don't.