9 ms·

What does it mean for a machine to “understand”?

59 points by stablemap 7y ago

modeless 7y ago
> When I ask Google “Who did IBM’s Deep Blue system defeat?” and it gives me an infobox with the answer “Kasparov” in big letters, it has correctly understood my question. Of course this understanding is limited. If I follow up my question to Google with “When?”, it gives me the dictionary definition of “when” — it doesn’t interpret my question as part of a dialogue. Google Search doesn't, but Google Assistant does. I posed the exact queries suggested by the article and the second query of simply the word "when" did give the correct answer (May 11 1997).
- cdirkx 7y ago
  I remember that when my friend got a Google Home almost 2 years ago, I was asking it some questions to explore the limitations. I asked about a certain restaurant chain, and it gave me the information, but then I asked "is there one near me". It listed all places with "one" in the name near me. I wonder if now it would correctly take the previous context into account. Google has been working a lot on improving their search and assistants to be "conversational". [1] looks like one of the results of this endevour. [1] https://cloud.google.com/dialogflow/docs/contexts-overview https://cloud.google.com/dialogflow/docs/contexts-overview
- baddox 7y ago
  That example seems pretty unrelated to what I would think of as “understanding.” That’s more just a feature request for Siri. It’s like saying “my calculator lets me type ’1 + 2 =’ and gives me the answer ‘3,’ so it seems to understand that question, but when I look at the calculator I see there’s no ‘sqrt’ button that would show me the square root of 3.” The fact that my basic calculator doesn’t have a “sqrt” button is pretty irrelevant to how well it “understands” how to add two numbers together.
  - taneq 7y ago
    Your basic calculator still has a concept of context, though. If you go '1 + 2 =' then it will give you '3', and if you press '/ 2 =' then it will give you '1.5'. It 'remembers what you were talking about' within its very limited scope. I think what they were trying to get at is that understanding is stateful.
igammarays 7y ago
I'm with John Searle on the Chinese room [1] opinion, i.e. that a machine cannot be said to "understand" language even if it is able to pass the Turing Test. That is because when we say "understand", we are referring to particular kind of human experience (qualia?) that a machine simply doesn't seem to have, but animals, for example, do. [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- baddox 7y ago
  It’s interesting that you said the machine doesn’t seem to have the experience, but animals do seem to have it. Can you explain what you mean by that?
  - igammarays 7y ago
    It’s a feeling. Understanding is a feeling, not just a logical set of inputs and outputs.
- msla 7y ago
  I can say that you don't have qualia and you can't prove me wrong. Does that seem dangerous to anyone else? I also don't see any distinction between "qualia" and "soul" other than spelling, but perhaps it's because I don't have one. Finally, I have this question for Searle: Say you understand English. Does any specific neuron in your brain understand English? No, the larger system of neurons+neuronal connections does, so why doesn't the system of grad student+book understand Chinese?
  - baddox 7y ago
    Dangerous? No. To me it just seems to mean that “qualia” is not a particularly useful concept, particularly when discussing the capabilities of computer software.
    - goatlover 7y ago
      The issue is that we’re using a word based on human mental activity and social agreement and then applying that to a computational process in a machine, which likely leaves out part of the human experience which makes up the word understand. It’s more accurate to say the Chinese room computes results which humans recognize as successful translation from English to Chinese. The understanding is all on the side interpreting the output.
basicplus2 7y ago
Self consciousness is required for understanding and intelligence
- dtech 7y ago
  why?
- yamrzou 7y ago
  How is that? Can you elaborate?
- sadness2 7y ago
  Self consciousness can be equally stratified and broken down functionally. It's not a boolean either.
- sadness2 7y ago
  I propose moral autonomy as a more interesting phenomena to distinguish.
js8 7y ago
I have a straightforward definition of "understand". To understand means to be able to give a (representative) example of the (intensionally) given set. Though it is harder than it seems, as it usually means solving the constraint satisfaction problem. For example, take the classical AI knowledgebase fragment, "bird is animal that flies". If I ask example of bird, it can say "eagle", and exhibit some understanding. We can then probe further and ask for a bird which is not an eagle. If it says "bat" or "balloon", it exhibits that it still doesn't understand birds quite right. In particular, if the description is nonsensical and thus impossible to understand, we cannot give any examples. This idea was really inspired by the study, where they asked people to recognize nonsensical and profound sentences, describing certain situation. The profound are the ones where you can create a concrete instance of the situation.
- yamrzou 7y ago
  Do you have a link to the mentioned study? Edit: nvm, I think I found it : http://journal.sjdm.org/15/15923a/jdm15923a.pdf http://journal.sjdm.org/15/15923a/jdm15923a.pdf
  - js8 7y ago
    It was on HN before: https://news.ycombinator.com/item?id=17764348 https://news.ycombinator.com/item?id=17764348 But perhaps I wasn't clear, the study doesn't say this, but it was rather my own experience with the BS sentences in that study that led me to the observation that they have an empty set of examples if we take them as a constraint satisfaction problem of sorts.
- deepbake 7y ago
  "bat" would be a correct response based on the knowledge fragment wouldn't it?
  - js8 7y ago
    I think you're right. The word "to understand" has two meanings. In the narrow sense, it's the feeling that we have when we "get it", that is we think that we have built a correct model of reality and it passes the logic consistency check (which is verified by being able to give an example or counterexample). In the broad sense, it means to build and apply correct models of reality. Above, I am talking in the narrow sense. So the fact that the model itself is wrong shouldn't be an issue. But in the broad sense, we could say that understanding is ability to convert between intensional and extensional (ostensive) representations (models) of the world. Finding an example from intensional representation is just one task that is required.
visarga 7y ago
Does AlphaGo 'understand' go? I think the key ingredient is 'being in the game', that means, having a body, being in an environment with a purpose. Humans are by default playing this game called 'life', we have to understand otherwise we perish, or our genes perish. It's not about symbolic vs connectionist, or qualia, or self consciousness. It's about being in the world, acting and observing the effects of actions, and having something to win or lose as a consequence of acting. This doesn't happen when training a neural net to recognise objects in images or doing translation. It's just a static dataset, a 'dead' world. AI until now has had a hard time simulating agents or creating real robotic bodies - it's expensive, and the system learns slowly, and it's unstable. But progress happens. Until our AI agents get real hands and feet and a purpose they can't be in the world and develop true understanding, they are more like subsystems of the brain than the whole brain. We need to close the loop with the environment for true understanding.
- goatlover 7y ago
  It certainly doesn’t understand Go as a board game humans invented as a stimulating mental exercise that became competitive enough to see whether human programmers could come up with a program that could beat any human. And whatever cultural history went along with playing Go. Certainly chess playing has been used as an analogy in the west for many activities involving strategy. This is something no computer currently understands.
  - visarga 7y ago
    It might not understand the socio-cultural context of the game, but it understands strategy better than us.
    - goatlover 7y ago
      Right, so computers are better at us for many tasks, but if we're thinking in terms of general intelligence, the context is pretty important.
- julvo 7y ago
nutanc 7y ago
This is a good balanced article that gets a lot of things right. We should take a forgiving approach when we talk about AI systems. And as the author points out the problem is not that AI systems dont have understanding yet. The problem is with the hype which leads many to believe that we are close to building systems which can understand us. That said, I have a small problem with the examples presented to say that already machines understand us :) The article says 'For example, when I tell Siri “Call Carol” and it dials the correct number, you will have a hard time convincing me that Siri did not understand my request" Let me try to take a shot at trying to explain that Siri did not "understand" your request. Siri was waiting for a command and executed the best command that matched. Which is, make a phone call. It did not understand what you meant because it did not take the whole environment into consideration. What if Carol was just in the other room. A human would maybe just shout "hey Carol, Thomas is asking you to come", instead of making a phone call. If listening to a request and executing a command is understanding, then computers have been understanding us for a long time. Even without the latest advances in AI.
- netsharc 7y ago
  So the next version of Siri can locate Carol's phone in the next room and will just beep her phone to tell her to see you. Of course that's still not understanding. The classic analogue is of course the Chinese room argument: https://en.m.wikipedia.org/wiki/Chinese_room https://en.m.wikipedia.org/wiki/Chinese_room
  - TheOtherHobbes 7y ago
    Which is an absolutely textbook example of begging the question. If you could make a machine pass the Turing test it might be intelligent - but no one has, and it's debatable if it's even possible, and it's even more debatable if, hype notwithstanding, the Turing test is even a good test of human-equivalent intelligence, because it ignores side channels that are fundamental to human communication, including tone of voice, posture, and facial expression. (Yes, people communicate over email/SMS. But no one communicates over email/SMS without an implied social context that hugely limits and simplifies the content of any conversation.) It's not the "call Carol" problem that needs to be solved. It's the "understand the entire world context well enough to know how to call Carol without being told - which includes being able to research information that isn't already available, and also includes edge cases like 'We went to Carol's funeral last week' and 'Carol had her phone stolen yesterday' and 'Carol is flying to Australia and won't be receiving messages for another 12 hours" and "Carol prefers FaceTime to WhatsApp." And so on. Ultimately your toy machine has to show evidence that it understands the entire world and can learn about it like a human can - which includes being able to do original research that isn't a simple literal Google search, parse humour, understand emotional responses and common cultural references, and follow standard social protocols. That's a much harder problem than having a vaguely plausible limited text-only conversation, whether it's in Chinese, English, or Swahili.
friendlybus 7y ago
I don't think it's possible for machines to understand. Numbers are meaningless, our human actions give them a useful function. All of the meaning a computer appears to provide is the preassigned values of layers and layers of programming work done by humans. Even today AI has a lot of human tagging and categorization that makes it useful. The idea that a new self- sustaining meaning generation can arise out of the interlocking mechanisms of a computer is an interesting one. As we see self driven car CEOs describe some of the most advanced systems we have, requiring to be run in controlled environments and balking at the infinite complexity of real life, are we really building computer systems that are anything more than an incredibly sophisticated loop?
- prvnsmpth 7y ago
  Well, what does it mean for humans to "understand"? Don't humans understand things by altering the state and connections of neurons in the brain? You could make the argument that the brain is also an "incredibly sophisticated loop". My point is that humans are also highly-sophisticated, biological machines, so if you say machines cannot "understand", you are making the same claim for humans as well.
  - friendlybus 7y ago
    Humans also squirt fluids around in their brains. Brains as machines is one of many ways to think about humans. Humans can conceive of and move past thoughts or concepts that would cause a machine to crash. I think more ideas describe human brains than being simply machines, though that idea is useful in places. Making the claim about what a human is in the absolute, is more about what you fill the unknown with than the nature of a human. Understanding is the difficult question. I would argue the understanding people want out of machines is the ability to generate, use and self-manage tools and that the machine knows the tool's place or context under a human value, story or intent and adapt to the implications of that higher order. That in the most exaggerated sense would be perceived as a machine that understands, but of course people mean different things when they say that.
    - 7y ago
RaiseProfits 7y ago
You should direct the question to the computer if you want a meaningful answer.
BoppreH 7y ago
I don't remember where I first saw it, but the best definition of "understanding" I've seen is "being able to encode and compress". For example, imagine a system that has as input the picture of a human face in RAW format. If the system runs the picture through JPEG compression, for example, and returns something substantially smaller, it has shown some understanding of the input (color, spatial repetition, etc). A more advanced system, with more understanding, may recognize it as a human face, and convert it to a template like the ones used for facial recognition. It doesn't care about individual pixels anymore, or the lighting, just general features of faces. It understands faces. An even more advanced system may recognize the specific person and compress the whole thing to a few bits. I would say that an OCR scanner understands the alphabet and how text is laid out, GPT-2 understands the relationship between words and how text is written. And a physics simulator understands basic physics because it can approximately compress a sequence of object movements into only initial conditions and small corrections. Lossy compression makes this concept non-trivial to measure, but it's still a world's away from the normal philosophical arguments.
boyadjian 7y ago
To understand means to classify, to modelize.
Entropee 7y ago
There is a side-note to be had here. Shannon defines 'communication' as 'all the procedures by which one mind may affect another and then goes on to define the semantic problem ( How precisely do the transmitted symbols convey the desired meaning? ) and the effectiveness problem ( How effectively does the received meaning affect the conduct in the desired way? ). In both cases it seems to me that 'intentionality' is a necessary (but insufficient) pre-requisite for understanding communicated messages. If the clouds in the sky happened to spell out "I am thirsty." you would notch this down to some ridiculous coincidence which satisfies the Infinite monkeys theorem. There is nothing to be understood in that sentence because there is neither an intended meaning nor an expectation of effectiveness. There is something to be understood when a human says it. When you bring that person a glass of water they have positive feedback loop that their message was effective.
cjfd 7y ago
On the one hand the quote by Edsger Dijkstra comes to mind. "The question of whether machines can think is about as relevant as the question of whether submarines can swim." We are hardwired to attribute great significance to what happens both in our own head and that of other people. On the other hand, machines still perform actions that one could call 'stupid'. When alphago was losing in the fourth match against Lee Sedol it would play 'stupid' moves. These were, for instance, trivial threads that any somewhat accomplished amateur go player would recognize in an instant and answer correctly. Humans, and also animals, have a hierarchy in their understanding of things. This maps on brain structure too. Evolution has added layers to the brain while keeping the existing structure. In this layered structure the lower parts are faster and more accurate but not as sophisticated. Stupidity arises because of a lack of layeredness so when the goal of winning the game is thwarted the top layer doesn't have any useful thing to do anymore and it falls back on a layer behind that. For alphago pretty much the only layer behind its very strong go engine is the rules of go. So, even when it is losing it will never play an illegal move but it will do otherwise trivially stupid things. For humans there is a layer between these things that prevents them from doing useless stuff. For living entities this is essential for survival. You can be forgetful of your dentist appointment but it is not possible to forget to let your heart beat. It seems that this problem could be mended by putting layers between the top level algorithm and most basic hardware level such that stupid stuff is preempted.
- AnIdiotOnTheNet 7y ago
  > When alphago was losing in the fourth match against Lee Sedol it would play 'stupid' moves. These were, for instance, trivial threads that any somewhat accomplished amateur go player would recognize in an instant and answer correctly. I think this behavior is less 'stupid' than it appears. When human beings play Go, the points matter even to the loser, and everyone goes home when it is over. There is life outside of Go. To Alpha Go, Go is it's entire universe. Part of the way it was trained was competing against other instances of itself, a sort of Thunderdome where the loser doesn't get to continue existing, and doesn't contribute to future generations. To Alpha Go, defeat is death. The behavior we observe when losing is nigh-certain has a human equivalent, we call it desperation. Alpha Go is trying moves that can only possibly work if the opponent makes a catastrophic blunder, which is incredibly unlikely, but it's the only shot it has.
deleted 7y ago
[deleted]
YeGoblynQueenne 7y ago
OK, wow, the old guard sure knows how to write sensibly. This is a great article. But I have to disagree with this (because of course I do): >> For example, when I tell Siri “Call Carol” and it dials the correct number, you will have a hard time convincing me that Siri did not understand my request. That is a very common-sense and down-to-earth non-definition of intelligence: how can an entity that is answering a question correctly not "understand" the question? I am going to quote Richard Feynman who encountered an example of this "how": After a lot of investigation, I finally figured out that the students had memorized everything, but they didn’t know what anything meant. When they heard “light that is reflected from a medium with an index,” they didn’t know that it meant a material such as water. They didn’t know that the “direction of the light” is the direction in which you see something when you’re looking at it, and so on. Everything was entirely memorized, yet nothing had been translated into meaningful words. So if I asked, “What is Brewster’s Angle?” I’m going into the computer with the right keywords. But if I say, “Look at the water,” nothing happens – they don’t have anything under “Look at the water”! https://v.cx/2010/04/feynman-brazil-education https://v.cx/2010/04/feynman-brazil-education In this (in?) famous passage Feynman is arguing that students of physics that he met in Brazil didn't know physics, even though they had memorised physics textbooks. Feynman doesn't talk about "understanding". Rather he talks about "knowing" a subject. But his is also a very straight-forward definition of knowing: you can tell whether someone knows a subject if you ask them many questions from different angles and find that they can only answer the questions asked from one single angle. So if I follow up "Siri, call Carol" with "Siri, what is a call" and Siri answers by calling Carol, I know that Siri doesn't know what a call is, probably doesn't know what a Carol is, or what a call-Carol is, and so that Siri doesn't have any understanding from a very common-sense point of view. Not sure if this goes beyond the Chinese room argument though. Perhaps I'm just on a diffferent side of it than Thomas Dietterich.
stared 7y ago
> Speaking as a psychologist, I’m flabbergasted by claims that the decisions of algorithms are opaque while the decisions of people are transparent. I’ve spent half my life at it and I still have limited success understanding human decisions. - Jean-François Bonnefon’s tweet (as quoted in https://p.migdal.pl/2019/07/15/human-machine-learning-motivation.html https://p.migdal.pl/2019/07/15/human-machine-learning-motiva...)
- gus_massa 7y ago
  The advantage of humans is that we have a building bullshit generator. If someone ask why you like ice-cream, you can tell a nice story about the hot summers during your childhood, but the reality is that sugar and fat are very useful. If a the autopilot of a Tesla hit someone, the error report is "Fatal error 0xDEADBEEF: coefficient 742 > 812". If a person hit someone the explanation is "It was dark and near a curve. I was texting that is totally safe. I got distracted by reindeer nearby. And I snoozed and was thinking about reaching a handkerchief".
Nasrudith 7y ago
To be gadflyish do humans even truly understand or do they just claim they do because they had the observations roughly encoded from what they have been taught? Teachings which themselves often include unfounded assumptions or outright superstition. Human understanding has been wrong often enough, missing enough crucial context to be dangerously hillariously wrong even amongst the "experts" of the day who came closest. The isn't some epistemological nilhism but to point out that understanding is incomplete for everyone and just because a given intelligence subset doesn't match with our assumptions doesn't mean it is wrong - although it also isn't always right.
avmich 7y ago
The proposal in the article is to define "understanding" and work towards testable satisfaction of the definition. This sounds a bit like a studying for a test taking. What if we made a definition and then worked successfully to reach the state when, according to this definition, the system "understands". Can we expect to be satisfied with the result in general, outside of the definition? The definition of understanding could be tricky, as history suggests. Other than "to understand is to translate into a form which is suitable for some use", there could be many definitions. Article itself brings examples of chess playing or truck driving which were considered good indicators, yet failed to satisfy us in some ways. Maybe we should just keep redefining "understanding" as good as we can today, and changing it if needed, and work trying to create a system "good", not necessarily "passing the test"?
ilaksh 7y ago
I think getting near human level for NLP understanding means be being able to visualize and combine all of the dynamic systems that language represents. I mean it's obvious that you can get pretty far just by processing a lot of text, but there is a limit. Some information about the way things work just is not encoded very well in text the way it is in video input. So you need to be able to do a sort of physics simulation for starters. Except it can't just be physics, because there are a lot of patterns that occur that you need to be able to call up and manipulate or combine that are not just plain physics. These patterns are not represented in text. There are projects doing video and text understanding. I think the trick to efficient generalization is to have the representations properly factored out somehow. Maybe things like capsule networks will help. Although that my guess is that to get really sort of componentized efficient understanding neural networks are not going to be the most effective way.

59 points by stablemap 7y ago

> When I ask Google “Who did IBM’s Deep Blue system defeat?” and it gives me an infobox with the answer “Kasparov” in big letters, it has correctly understood my question. Of course this understanding is limited. If I follow up my question to Google with “When?”, it gives me the dictionary definition of “when” — it doesn’t interpret my question as part of a dialogue. Google Search doesn't, but Google Assistant does. I posed the exact queries suggested by the article and the second query of simply the word "when" did give the correct answer (May 11 1997).

cdirkx 7y ago
I remember that when my friend got a Google Home almost 2 years ago, I was asking it some questions to explore the limitations. I asked about a certain restaurant chain, and it gave me the information, but then I asked "is there one near me". It listed all places with "one" in the name near me. I wonder if now it would correctly take the previous context into account. Google has been working a lot on improving their search and assistants to be "conversational". [1] looks like one of the results of this endevour. [1] https://cloud.google.com/dialogflow/docs/contexts-overview https://cloud.google.com/dialogflow/docs/contexts-overview
baddox 7y ago
That example seems pretty unrelated to what I would think of as “understanding.” That’s more just a feature request for Siri. It’s like saying “my calculator lets me type ’1 + 2 =’ and gives me the answer ‘3,’ so it seems to understand that question, but when I look at the calculator I see there’s no ‘sqrt’ button that would show me the square root of 3.” The fact that my basic calculator doesn’t have a “sqrt” button is pretty irrelevant to how well it “understands” how to add two numbers together.
- taneq 7y ago
  Your basic calculator still has a concept of context, though. If you go '1 + 2 =' then it will give you '3', and if you press '/ 2 =' then it will give you '1.5'. It 'remembers what you were talking about' within its very limited scope. I think what they were trying to get at is that understanding is stateful.

igammarays 7y ago

I'm with John Searle on the Chinese room [1] opinion, i.e. that a machine cannot be said to "understand" language even if it is able to pass the Turing Test. That is because when we say "understand", we are referring to particular kind of human experience (qualia?) that a machine simply doesn't seem to have, but animals, for example, do. [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

baddox 7y ago
It’s interesting that you said the machine doesn’t seem to have the experience, but animals do seem to have it. Can you explain what you mean by that?
- igammarays 7y ago
  It’s a feeling. Understanding is a feeling, not just a logical set of inputs and outputs.
msla 7y ago
I can say that you don't have qualia and you can't prove me wrong. Does that seem dangerous to anyone else? I also don't see any distinction between "qualia" and "soul" other than spelling, but perhaps it's because I don't have one. Finally, I have this question for Searle: Say you understand English. Does any specific neuron in your brain understand English? No, the larger system of neurons+neuronal connections does, so why doesn't the system of grad student+book understand Chinese?
- baddox 7y ago
  Dangerous? No. To me it just seems to mean that “qualia” is not a particularly useful concept, particularly when discussing the capabilities of computer software.
  - goatlover 7y ago
    The issue is that we’re using a word based on human mental activity and social agreement and then applying that to a computational process in a machine, which likely leaves out part of the human experience which makes up the word understand. It’s more accurate to say the Chinese room computes results which humans recognize as successful translation from English to Chinese. The understanding is all on the side interpreting the output.

basicplus2 7y ago

Self consciousness is required for understanding and intelligence

dtech 7y ago
why?
yamrzou 7y ago
How is that? Can you elaborate?
sadness2 7y ago
Self consciousness can be equally stratified and broken down functionally. It's not a boolean either.
sadness2 7y ago
I propose moral autonomy as a more interesting phenomena to distinguish.

js8 7y ago

I have a straightforward definition of "understand". To understand means to be able to give a (representative) example of the (intensionally) given set. Though it is harder than it seems, as it usually means solving the constraint satisfaction problem. For example, take the classical AI knowledgebase fragment, "bird is animal that flies". If I ask example of bird, it can say "eagle", and exhibit some understanding. We can then probe further and ask for a bird which is not an eagle. If it says "bat" or "balloon", it exhibits that it still doesn't understand birds quite right. In particular, if the description is nonsensical and thus impossible to understand, we cannot give any examples. This idea was really inspired by the study, where they asked people to recognize nonsensical and profound sentences, describing certain situation. The profound are the ones where you can create a concrete instance of the situation.

yamrzou 7y ago
Do you have a link to the mentioned study? Edit: nvm, I think I found it : http://journal.sjdm.org/15/15923a/jdm15923a.pdf http://journal.sjdm.org/15/15923a/jdm15923a.pdf
- js8 7y ago
  It was on HN before: https://news.ycombinator.com/item?id=17764348 https://news.ycombinator.com/item?id=17764348 But perhaps I wasn't clear, the study doesn't say this, but it was rather my own experience with the BS sentences in that study that led me to the observation that they have an empty set of examples if we take them as a constraint satisfaction problem of sorts.
deepbake 7y ago
"bat" would be a correct response based on the knowledge fragment wouldn't it?
- js8 7y ago
  I think you're right. The word "to understand" has two meanings. In the narrow sense, it's the feeling that we have when we "get it", that is we think that we have built a correct model of reality and it passes the logic consistency check (which is verified by being able to give an example or counterexample). In the broad sense, it means to build and apply correct models of reality. Above, I am talking in the narrow sense. So the fact that the model itself is wrong shouldn't be an issue. But in the broad sense, we could say that understanding is ability to convert between intensional and extensional (ostensive) representations (models) of the world. Finding an example from intensional representation is just one task that is required.

visarga 7y ago

Does AlphaGo 'understand' go? I think the key ingredient is 'being in the game', that means, having a body, being in an environment with a purpose. Humans are by default playing this game called 'life', we have to understand otherwise we perish, or our genes perish. It's not about symbolic vs connectionist, or qualia, or self consciousness. It's about being in the world, acting and observing the effects of actions, and having something to win or lose as a consequence of acting. This doesn't happen when training a neural net to recognise objects in images or doing translation. It's just a static dataset, a 'dead' world. AI until now has had a hard time simulating agents or creating real robotic bodies - it's expensive, and the system learns slowly, and it's unstable. But progress happens. Until our AI agents get real hands and feet and a purpose they can't be in the world and develop true understanding, they are more like subsystems of the brain than the whole brain. We need to close the loop with the environment for true understanding.

goatlover 7y ago
It certainly doesn’t understand Go as a board game humans invented as a stimulating mental exercise that became competitive enough to see whether human programmers could come up with a program that could beat any human. And whatever cultural history went along with playing Go. Certainly chess playing has been used as an analogy in the west for many activities involving strategy. This is something no computer currently understands.
- visarga 7y ago
  It might not understand the socio-cultural context of the game, but it understands strategy better than us.
  - goatlover 7y ago
    Right, so computers are better at us for many tasks, but if we're thinking in terms of general intelligence, the context is pretty important.
julvo 7y ago

nutanc 7y ago

This is a good balanced article that gets a lot of things right. We should take a forgiving approach when we talk about AI systems. And as the author points out the problem is not that AI systems dont have understanding yet. The problem is with the hype which leads many to believe that we are close to building systems which can understand us. That said, I have a small problem with the examples presented to say that already machines understand us :) The article says 'For example, when I tell Siri “Call Carol” and it dials the correct number, you will have a hard time convincing me that Siri did not understand my request" Let me try to take a shot at trying to explain that Siri did not "understand" your request. Siri was waiting for a command and executed the best command that matched. Which is, make a phone call. It did not understand what you meant because it did not take the whole environment into consideration. What if Carol was just in the other room. A human would maybe just shout "hey Carol, Thomas is asking you to come", instead of making a phone call. If listening to a request and executing a command is understanding, then computers have been understanding us for a long time. Even without the latest advances in AI.

netsharc 7y ago
So the next version of Siri can locate Carol's phone in the next room and will just beep her phone to tell her to see you. Of course that's still not understanding. The classic analogue is of course the Chinese room argument: https://en.m.wikipedia.org/wiki/Chinese_room https://en.m.wikipedia.org/wiki/Chinese_room
- TheOtherHobbes 7y ago
  Which is an absolutely textbook example of begging the question. If you could make a machine pass the Turing test it might be intelligent - but no one has, and it's debatable if it's even possible, and it's even more debatable if, hype notwithstanding, the Turing test is even a good test of human-equivalent intelligence, because it ignores side channels that are fundamental to human communication, including tone of voice, posture, and facial expression. (Yes, people communicate over email/SMS. But no one communicates over email/SMS without an implied social context that hugely limits and simplifies the content of any conversation.) It's not the "call Carol" problem that needs to be solved. It's the "understand the entire world context well enough to know how to call Carol without being told - which includes being able to research information that isn't already available, and also includes edge cases like 'We went to Carol's funeral last week' and 'Carol had her phone stolen yesterday' and 'Carol is flying to Australia and won't be receiving messages for another 12 hours" and "Carol prefers FaceTime to WhatsApp." And so on. Ultimately your toy machine has to show evidence that it understands the entire world and can learn about it like a human can - which includes being able to do original research that isn't a simple literal Google search, parse humour, understand emotional responses and common cultural references, and follow standard social protocols. That's a much harder problem than having a vaguely plausible limited text-only conversation, whether it's in Chinese, English, or Swahili.

friendlybus 7y ago

I don't think it's possible for machines to understand. Numbers are meaningless, our human actions give them a useful function. All of the meaning a computer appears to provide is the preassigned values of layers and layers of programming work done by humans. Even today AI has a lot of human tagging and categorization that makes it useful. The idea that a new self- sustaining meaning generation can arise out of the interlocking mechanisms of a computer is an interesting one. As we see self driven car CEOs describe some of the most advanced systems we have, requiring to be run in controlled environments and balking at the infinite complexity of real life, are we really building computer systems that are anything more than an incredibly sophisticated loop?

prvnsmpth 7y ago
Well, what does it mean for humans to "understand"? Don't humans understand things by altering the state and connections of neurons in the brain? You could make the argument that the brain is also an "incredibly sophisticated loop". My point is that humans are also highly-sophisticated, biological machines, so if you say machines cannot "understand", you are making the same claim for humans as well.
- friendlybus 7y ago
  Humans also squirt fluids around in their brains. Brains as machines is one of many ways to think about humans. Humans can conceive of and move past thoughts or concepts that would cause a machine to crash. I think more ideas describe human brains than being simply machines, though that idea is useful in places. Making the claim about what a human is in the absolute, is more about what you fill the unknown with than the nature of a human. Understanding is the difficult question. I would argue the understanding people want out of machines is the ability to generate, use and self-manage tools and that the machine knows the tool's place or context under a human value, story or intent and adapt to the implications of that higher order. That in the most exaggerated sense would be perceived as a machine that understands, but of course people mean different things when they say that.
  - 7y ago

RaiseProfits 7y ago

You should direct the question to the computer if you want a meaningful answer.

BoppreH 7y ago

I don't remember where I first saw it, but the best definition of "understanding" I've seen is "being able to encode and compress". For example, imagine a system that has as input the picture of a human face in RAW format. If the system runs the picture through JPEG compression, for example, and returns something substantially smaller, it has shown some understanding of the input (color, spatial repetition, etc). A more advanced system, with more understanding, may recognize it as a human face, and convert it to a template like the ones used for facial recognition. It doesn't care about individual pixels anymore, or the lighting, just general features of faces. It understands faces. An even more advanced system may recognize the specific person and compress the whole thing to a few bits. I would say that an OCR scanner understands the alphabet and how text is laid out, GPT-2 understands the relationship between words and how text is written. And a physics simulator understands basic physics because it can approximately compress a sequence of object movements into only initial conditions and small corrections. Lossy compression makes this concept non-trivial to measure, but it's still a world's away from the normal philosophical arguments.

boyadjian 7y ago

To understand means to classify, to modelize.

Entropee 7y ago

There is a side-note to be had here. Shannon defines 'communication' as 'all the procedures by which one mind may affect another and then goes on to define the semantic problem ( How precisely do the transmitted symbols convey the desired meaning? ) and the effectiveness problem ( How effectively does the received meaning affect the conduct in the desired way? ). In both cases it seems to me that 'intentionality' is a necessary (but insufficient) pre-requisite for understanding communicated messages. If the clouds in the sky happened to spell out "I am thirsty." you would notch this down to some ridiculous coincidence which satisfies the Infinite monkeys theorem. There is nothing to be understood in that sentence because there is neither an intended meaning nor an expectation of effectiveness. There is something to be understood when a human says it. When you bring that person a glass of water they have positive feedback loop that their message was effective.

cjfd 7y ago

On the one hand the quote by Edsger Dijkstra comes to mind. "The question of whether machines can think is about as relevant as the question of whether submarines can swim." We are hardwired to attribute great significance to what happens both in our own head and that of other people. On the other hand, machines still perform actions that one could call 'stupid'. When alphago was losing in the fourth match against Lee Sedol it would play 'stupid' moves. These were, for instance, trivial threads that any somewhat accomplished amateur go player would recognize in an instant and answer correctly. Humans, and also animals, have a hierarchy in their understanding of things. This maps on brain structure too. Evolution has added layers to the brain while keeping the existing structure. In this layered structure the lower parts are faster and more accurate but not as sophisticated. Stupidity arises because of a lack of layeredness so when the goal of winning the game is thwarted the top layer doesn't have any useful thing to do anymore and it falls back on a layer behind that. For alphago pretty much the only layer behind its very strong go engine is the rules of go. So, even when it is losing it will never play an illegal move but it will do otherwise trivially stupid things. For humans there is a layer between these things that prevents them from doing useless stuff. For living entities this is essential for survival. You can be forgetful of your dentist appointment but it is not possible to forget to let your heart beat. It seems that this problem could be mended by putting layers between the top level algorithm and most basic hardware level such that stupid stuff is preempted.

AnIdiotOnTheNet 7y ago
> When alphago was losing in the fourth match against Lee Sedol it would play 'stupid' moves. These were, for instance, trivial threads that any somewhat accomplished amateur go player would recognize in an instant and answer correctly. I think this behavior is less 'stupid' than it appears. When human beings play Go, the points matter even to the loser, and everyone goes home when it is over. There is life outside of Go. To Alpha Go, Go is it's entire universe. Part of the way it was trained was competing against other instances of itself, a sort of Thunderdome where the loser doesn't get to continue existing, and doesn't contribute to future generations. To Alpha Go, defeat is death. The behavior we observe when losing is nigh-certain has a human equivalent, we call it desperation. Alpha Go is trying moves that can only possibly work if the opponent makes a catastrophic blunder, which is incredibly unlikely, but it's the only shot it has.

deleted 7y ago

[deleted]

YeGoblynQueenne 7y ago

OK, wow, the old guard sure knows how to write sensibly. This is a great article. But I have to disagree with this (because of course I do): >> For example, when I tell Siri “Call Carol” and it dials the correct number, you will have a hard time convincing me that Siri did not understand my request. That is a very common-sense and down-to-earth non-definition of intelligence: how can an entity that is answering a question correctly not "understand" the question? I am going to quote Richard Feynman who encountered an example of this "how": After a lot of investigation, I finally figured out that the students had memorized everything, but they didn’t know what anything meant. When they heard “light that is reflected from a medium with an index,” they didn’t know that it meant a material such as water. They didn’t know that the “direction of the light” is the direction in which you see something when you’re looking at it, and so on. Everything was entirely memorized, yet nothing had been translated into meaningful words. So if I asked, “What is Brewster’s Angle?” I’m going into the computer with the right keywords. But if I say, “Look at the water,” nothing happens – they don’t have anything under “Look at the water”! https://v.cx/2010/04/feynman-brazil-education https://v.cx/2010/04/feynman-brazil-education In this (in?) famous passage Feynman is arguing that students of physics that he met in Brazil didn't know physics, even though they had memorised physics textbooks. Feynman doesn't talk about "understanding". Rather he talks about "knowing" a subject. But his is also a very straight-forward definition of knowing: you can tell whether someone knows a subject if you ask them many questions from different angles and find that they can only answer the questions asked from one single angle. So if I follow up "Siri, call Carol" with "Siri, what is a call" and Siri answers by calling Carol, I know that Siri doesn't know what a call is, probably doesn't know what a Carol is, or what a call-Carol is, and so that Siri doesn't have any understanding from a very common-sense point of view. Not sure if this goes beyond the Chinese room argument though. Perhaps I'm just on a diffferent side of it than Thomas Dietterich.

stared 7y ago

> Speaking as a psychologist, I’m flabbergasted by claims that the decisions of algorithms are opaque while the decisions of people are transparent. I’ve spent half my life at it and I still have limited success understanding human decisions. - Jean-François Bonnefon’s tweet (as quoted in https://p.migdal.pl/2019/07/15/human-machine-learning-motivation.html https://p.migdal.pl/2019/07/15/human-machine-learning-motiva...)

gus_massa 7y ago
The advantage of humans is that we have a building bullshit generator. If someone ask why you like ice-cream, you can tell a nice story about the hot summers during your childhood, but the reality is that sugar and fat are very useful. If a the autopilot of a Tesla hit someone, the error report is "Fatal error 0xDEADBEEF: coefficient 742 > 812". If a person hit someone the explanation is "It was dark and near a curve. I was texting that is totally safe. I got distracted by reindeer nearby. And I snoozed and was thinking about reaching a handkerchief".

Nasrudith 7y ago

To be gadflyish do humans even truly understand or do they just claim they do because they had the observations roughly encoded from what they have been taught? Teachings which themselves often include unfounded assumptions or outright superstition. Human understanding has been wrong often enough, missing enough crucial context to be dangerously hillariously wrong even amongst the "experts" of the day who came closest. The isn't some epistemological nilhism but to point out that understanding is incomplete for everyone and just because a given intelligence subset doesn't match with our assumptions doesn't mean it is wrong - although it also isn't always right.

avmich 7y ago

The proposal in the article is to define "understanding" and work towards testable satisfaction of the definition. This sounds a bit like a studying for a test taking. What if we made a definition and then worked successfully to reach the state when, according to this definition, the system "understands". Can we expect to be satisfied with the result in general, outside of the definition? The definition of understanding could be tricky, as history suggests. Other than "to understand is to translate into a form which is suitable for some use", there could be many definitions. Article itself brings examples of chess playing or truck driving which were considered good indicators, yet failed to satisfy us in some ways. Maybe we should just keep redefining "understanding" as good as we can today, and changing it if needed, and work trying to create a system "good", not necessarily "passing the test"?

ilaksh 7y ago

I think getting near human level for NLP understanding means be being able to visualize and combine all of the dynamic systems that language represents. I mean it's obvious that you can get pretty far just by processing a lot of text, but there is a limit. Some information about the way things work just is not encoded very well in text the way it is in video input. So you need to be able to do a sort of physics simulation for starters. Except it can't just be physics, because there are a lot of patterns that occur that you need to be able to call up and manipulate or combine that are not just plain physics. These patterns are not represented in text. There are projects doing video and text understanding. I think the trick to efficient generalization is to have the representations properly factored out somehow. Maybe things like capsule networks will help. Although that my guess is that to get really sort of componentized efficient understanding neural networks are not going to be the most effective way.