6 ms·

We have zero evidence that these models have any “understanding” of anything. “Understanding” would mean that they be able to train themselves, which they are

• huytersd 3y ago
  No, you can have abstract representations of a concept which would fit understanding in a certain sense of the word. You can have “understanding” of a concept without an overarching self aware hypervisor. It’s like isolating a set of neurons in your brain that represent an idea.
• visarga 3y ago
  So, in order for a kid to understand multiplication means they must be able to train themselves and not regurgitate the multiplication table?
  • ipaddr 3y ago
    Yes I think so. Understanding would involve understanding that it's a short form for adding. Remembering multiplication tables allows you to use math but it doesn't infer understanding. Training themselves is necessary. All learning is self learning. Teachers can present material in different ways but learning is personal. No one can force you to learn either.
    • visarga 3y ago
      You can check if a model (or a kid) understands multiplication by simply probing them with questions, to explain the concept in their own words, or on concrete examples. If their answers are robust they understand, if their answers are fragile and very input dependent, they don't. "To understand" is one of those poorly defined concepts, like "consciousness", it is thrown a lot in the face when talking about AI. But what does it mean actually? It means to have a working model of the thing you are understanding, a causal model that adapts to any new configuration of the inputs reliably. Or in other words it means to generalize well around that topic. The opposite would be to "learn to the test" or "overfit the problem" and only be able to solve very limited cases that follow the training pattern closely. That would make for brittle learning, at surface level, based on shortcuts.
      • int_19h 3y ago
        > It means to have a working model of the thing you are understanding, a causal model that adapts to any new configuration of the inputs reliably The weasel word here is "reliably". What does this actually mean? It obviously cannot be reliable in a sense of always giving the correct result, because this would make understanding something a strict binary, and we definitely don't treat it like that for humans - we say things like "they understand it better than me" all the time, which when you boil it down has to mean "their model of it is more predictive than mine". But then if that is a quantifiable measure, then we're really talking about "reliable enough". And then the questions are: 1) where do you draw that line, exactly, and 2) even more importantly, why do you draw the line there and not somewhere else. For me, the only sensible answer to this is to refuse to draw the line at all, and just embrace the fact that understanding is a spectrum. But then it doesn't even make sense to ask questions like "does the model really understands?" - they are meaningless. (The same goes for concepts like "consciousness" or "intelligence", by the way.) The reason why I think this isn't universally accepted is because it makes us not special, and humans really, really like to think of themselves as special (just look at our religions).
        • haswell 3y ago
          > It obviously cannot be reliable in a sense of always giving the correct result, because this would make understanding something a strict binary Our capacity to make mistakes does not necessarily equate to a lack of understanding. If you’re doing a difficult math problem and get it wrong, that doesn’t necessarily imply that you don’t understand the problem. It speaks to a limitation of our problem solving machinery and the implements we use to carry out tasks. e.g. if I’m not paying close enough attention and write down the wrong digit in the middle of solving a problem, that could also just be because I got distracted, or made a mistake. If I did the same problem again from scratch, I would probably get it right if I understand the subject matter. Limitations of our working memory, how distracted we are that day, mis-keying something on a calculator or writing down the wrong digit, etc. can all lead to a wrong answer. This is distinct from encountering a problem where one’s understanding was incomplete leading to consistently wrong answers. There are clearly people who are better and worse comparatively at solving certain problems. But given the complexity of our brains/biology, there are myriad reasons for these differences. Clearly there are people who have the capacity to understand certain problems more deeply (e.g. Einstein), but all of this was primarily to say that output doesn’t need to be 100% “reliable” to imply a complete understanding.
  • mardef 3y ago
    I wouldn't call someone reciting a French-English dictionary a French speaker that understands the language. Other way around. If a kid understood the concepts of multiplation, they could train themselves on the next logical steps like exponents. The consequences of this in the terms of AI would mean they build on a series of concepts and would quickly dwarf us in intelligence.
    • kenjackson 3y ago
      How many people years existed between the time people discovered multiplication and then exponentiation? Did it happen in wall clock time in a single generation?
      • wizzwizz4 3y ago
        > How many people years existed between the time people discovered multiplication and then exponentiation? We don't know, because these concepts were discovered before writing. We do know that far larger jumps have been made by individual mathematicians who never made it past 33 years of age.
    • int_19h 3y ago
      Neither would I. But if someone who can translate a French sentence into English preserving the meaning most of the time, I think it would be reasonable to say that this person understands French. And that is what we're talking about here - models do produce correct output much of the time, even when you give them input that was not present in their training data. The "stochastic parrot" argument is precisely the claim that this still does not prove understanding. The "Chinese room" argument takes it one notch further and claims that even if the translation is 100% correct 100% of the time, that still doesn't prove understanding.
• esafak 3y ago
  I don't buy that definition of understanding. Yours is closer to learning. If you had an accident and lost the ability to learn new things, you would still be able to understand based on what you have learnt so far. And that's what these models are like. They don't retrain themselves because we haven't told them to.
• chaxor 3y ago
  While I agree with the hidden statement of the utility in online reinforcement learning here, it should be pointed out that for some snapshot of a system, which may have already been trained with a large amount of data - the structure of the learned space and the inferences within it seem like a reasonable definition for 'understanding'. I don't know many people that would suggest that knowing about ones family structure, such as their mom/dad/uncle, and how their history relates to them, is required to be _completely reconstructed_ every time they interact with their environment somehow from first principles. Online reinforcement learning can have large merits without resorting to stating it's required for learning. Just as self awareness can occur independently of consciousness can occur independently of intelligence is independent of empathy and so on. They are all different, and having one component doesn't mean anything about the rest.
• xetplan 3y ago
  We don't need evidence anymore. We are just so fucking brilliant collectively we could never be fooled by nature again!