7 ms·

Why Deep Learning surprises me

komaromy 9y ago
> Computers understand things as well as us, perhaps better. If this was limited to chess, I would unquestionably agree. If it was limited to image recognition, I would tentatively agree, although things like [0] make me cautious (admittedly, that was from March, and I'm not familiar with progress since then). However, the author seems to be generalizing beyond those two domains, to the limits of human understanding. That seems like a couple-orders-of-magnitude leap too far to me. For example, I don't know of any autonomous system capable of understanding a short novel with simple language and writing a one-page summary of it, as might be expected of a human ten-year-old. [0] https://twitter.com/Meaningness/status/846478348947668992 https://twitter.com/Meaningness/status/846478348947668992
- amelius 9y ago
  Perhaps your reference [0] would work on the human brain too, if only we could know all the weights assigned to all neurons/axons of the given human this should apply to :)
  - komaromy 9y ago
    Could be! We'll need a volunteer comfortable with having their neuronal weights experimented on.
    - bitL 9y ago
      You are forgetting local protein-based computations observed in biological neurons we have no clue about...
  - statusgraph 9y ago
    Interestingly, you can treat the NN as a black box (ie, not look at individual weights or even the architecture) and still derive adversarial cases: https://arxiv.org/abs/1602.02697 https://arxiv.org/abs/1602.02697
    - amelius 9y ago
      Interesting. Would that work on humans too?
- bitL 9y ago
  To what extent those universal perturbations are causing problems due to insufficient image augmentation? Or due to deficient optimizer used while training CNNs (all optimizers are just heuristics with nasty failure cases)? Could we train a GAN-like DNN on those perturbations to make their effect disappear?
singham 9y ago
Daniel Dennett has been saying this for quite a while.
bitL 9y ago
I think author is stretching arguments here a bit - DL is just partitioning space according to some pre-baked associations given to it during training; in this case it's more like a non-linear optimization where we want to end up with N-million dimensional objects of certain shape obtained by optimizing some objective function allowing predicting similar associations. It doesn't have much with the actual innate quality of understanding. Maybe reinforcement learning with deep learning together (DRL) can move us towards such a quality at least in a mechanical sense.
- petters 9y ago
  On one hand, I sort of agree with you. On the other hand, what you are saying feels a little bit like saying that humans aren't impressive, because we are just atoms. Sometimes interesting things arise from many small, simple parts.
- jacquesm 9y ago
  That's got to be one of the most concise but still complete descriptions of deep learning that I've seen so far. The question that it implicitly raises (at least, with me) is how can we tell the difference between 'understanding' and 'deep learning' if the end results are the same? To me 'reasoning' is a slow, conscious process, and understanding is a part of that. But classification problems , especially when done by humans when they try to work fast have no room for such conscious decision making, we go much faster than that and outsource the job to our subconscious. Predictably, the error rate goes up and in those kind of situations deep learning can today already outperform humans on the same tasks. The weird thing is that deep learning solutions can get simple cases completely wrong, where a human would never err, and yet get some of the hardest cases - where a human would be very likely to make an error - right. It's baffling.
- akyu 9y ago
  >It doesn't have much with the actual innate quality of understanding That was his entire point.
- westoncb 9y ago
  > DL is just partitioning space according to some pre-baked associations given to it during training What I wonder is whether that's not also maybe the cornerstone of human understanding. If I understand correctly, you are essentially saying that DL is forming categories, or developing a classification scheme. Granted, if we're only talking about supervised DL, and the program is practically told where to form the boundaries—then it's not very impressive. But if the software is extracting statistically prominent commonalities and using those to form category boundaries, and arranging them hierarchically—then while the implementation may be totally different from human understanding, the effect seems to strongly overlap. I assume I'm probably just missing something—anyone know what it is? (It seems clear that at least part of the problem here is that 'human understanding' has been left far more vague than DL, and in order to say one way or the other how much they have in common, we need to better define 'human understanding'.)
mathgenius 9y ago
I don't see why "understanding" is equivalent to mere pattern recognition. Even using this word "recognition", what does that mean? It's another word like "understand". These algorithms are just pattern patterning. They don't even know they are patterning, that is a meta-property assigned in (or by) a context.
- ffwd 9y ago
  I agree, but I think human knowledge can be represented as either a graph, hierarchy or network of patterns. Like my knowledge of the letter 'A' is a network of connections to patterns 'language', 'english', 'alphabet', whatever else, and if the computer can do the same, it can use that knowledge of that network (as a whole separate entity) to make a decision, so to speak. Consciousness does come into it since we have a pretty visceral sense of it, and especially when we mentally trawl through our patterns to make some story, but really understanding should just be creating new patterns from existing patterns and the ability to utilize them as distinct entities in some way (rather than being emergent in the system implicitly and only being utilized by accident, say as emergent behavior randomly occurring because of local constraints)
- AndrewKemendo 9y ago
  I don't see why "understanding" is equivalent to mere pattern recognition. You're underestimating what goes into high accuracy pattern recognition as well as assuming that patterns exist for only one vector and in a single context. If I asked you to explain how you "understand" some concept, it will inevitably be how the structure and mechanics of it relate to others and in what context. All of those are simply patterns that are abstracted or made more granular. For example, how do you "understand" what a car is? You would inevitably describe some definition of a car mechanically and the context in which a car operates. So it's a contained combination of metal and plastic objects and usually liquids with a mechanism to transfer power through gearing and wheels, a compartment for humans, some control mechanisms etc... (definition of the technical), but it can't operate in water (boat) or in the air (airplane). Each of these things is learned through exposure over time, and recognized as connected, to come up with a "understanding" of a car even before it's formally defined. This is why children ask if cars can fly or go in the water.
AndrewOMartin 9y ago
Searle's Chinese Room Argument was specifically aimed at people claiming an algorithm could understand something because of its behaviour. It applies to Deep Learning as much as it does Schank and Ableson's script understanding system.
- nightski 9y ago
  If a human can translate perfectly without understanding the conversation, then that to me implies that the mind itself gives no innate intelligence similar to the computer. It must be taught the meaning of things, exactly as a computer would need to be. I'm just not following his logic, it feels like a straw man. Of course the computer doesn't understand the meaning of the symbols it is translating, because it was never given data to teach it that (similar to a human in the scenario).
- inventtheday 9y ago
  The Chinese Room Argument is deeply flawed because it assumes that language translation in humans is a conscious phenomenon. In fact, if you're proficient in a foreign language, you can relate to the fact that for the most part translation happens in the black box of the subconscious mind. The words "bubble out" naturally. The black box of the subconscious mind is no different than the black box of the Chinese room. "Understanding" in the traditional sense is absent from both processes.
dna_polymerase 9y ago
> Given enough examples, computers can understand what is letter "A" and what is letter "B". Meh. Given enough examples, computers now can distinguish letter A and B but distinguishing is not understanding. You could argue that after learning the Network just uses an instruction set and from the outside that may leave the impression of understanding but it really does not. Isn't that basically the Chinese room thing?
- hyperbovine 9y ago
  In fact recent research indicates that you can randomly relabel the training examples and the network still achieves zero training error (https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530). So it is not "understanding" anything intrinsic or fundamental about the letter "A". Rather, it's just storing training examples somewhere inside of its millions of parameters, which sounds a lot less impressive.
  - jimfleming 9y ago
    That is not a conclusion that can be drawn from the findings in the paper. While the models they evaluate can achieve zero training error on random labels, the test error is obviously not zero: it doesn't generalize at all. However, training on real labels often finds solutions which can generalize quite well. A better way to summarize the central question of this paper would be: "Why is it that a large-parameter model trained with gradient descent on real data _could_ just memorize all of the training data (it has the capacity) yet finds solutions which generalize well to an unseen test set?" To say that deep learning is _just_ memorizing its training data would be incorrect. We have empirical evidence to the contrary and this paper is part of that evidence.
    - hyperbovine 9y ago
      But we also have empirical evidence that they generalize incredible poorly, namely the existence of imperceptible (adversarial) perturbations which can transfer across images and networks and are catastrophically misclassified.
jeremynixon 9y ago
The mysticism around ‘Emergence’ is just a modeling error where people only abstract in one way (say, down to cells) and don’t include something important like the interaction between cells in their reductionist model of the system. It’s like creating a graph without the edges. And so when those effects have manifest consequences at a higher level, it feels like they appeared as if by magic.
- ktRolster 9y ago
  IT's kind of like saying, "This river is not the same river that it was upstream, and yet it is. The river is not the same water of last year, and yet it is the same river." The phenomenon is entirely well understood by all involved, and yet coming up with a reasonable definition is hard. http://existentialcomics.com/comic/164 http://existentialcomics.com/comic/164 So it's easier to be mystical.
mannykannot 9y ago
I have always believed that understanding is an emergent property of physical processes that could be modeled computationally, but I do not think deep learning has yet demonstrated that it has yet achieved it. Some of the evidence comes from the ways it fails, such as 'recognizing' images that humans would understand are not what the systems think they are, and being confident in decisions that make no sense. These situations occur precisely because of a lack of understanding. I am open to the possibility that deep learning alone might achieve understanding, but I think it is more likely to succumb to the law of diminishing returns before it gets there.
inventtheday 9y ago
Actually, computers are conscious as well. Consciousness is simply a system of information that operates on a continuous sense/plan/act loop. You could argue that they are "less" conscious, but to say that they are unconscious is to make the same mistake as people have made for years by saying that computers cannot "understand" anything. Some people push back on this by saying computers have no sense of self. Thats not true. Most computers do have internal state representations about themselves. Take a driverless car for example. When it does localization, it's constantly referencing its own shape and speed and comparing it to the environment. That's a sense of self. Whatever philosophical barriers we place between ourselves and machines (and animals/nature for that matter), one thing is for certain: they will eventually debunked.
- gfodor 9y ago
  Your last claim may be true, but your assertions don't constitute any form of debunking. We don't have an objectively agreed upon way to measure consciousness (though some have been proposed) so making bold claims like "computers are conscious as well" doesn't make much sense until we agree on a way to measure and experiment on the presence of consciousness.
  - inventtheday 9y ago
    Society has assumed the defacto circular definition of consciousness as "whatever we, as humans, are experiencing". For obvious reasons, this is not a helpful concept. Instead, I like to think of consciousness in terms of structures and mechanisms of information flow. If we open our minds up to this type of thinking, we can see consciousness in varying degrees in nature, in computers, and of course in people. For anyone who's curious, the guiding light in this school of thought is Hofstadter's Godel Escher Bach.
    - carapace 9y ago
      I upvoted your original comment above even though I don't agree because you're commenting in good faith in my opinion. Society as a whole has not assumed a concrete definition for consciousness, as there are a lot of people who don't give it a second thought. Among those who do recognize consciousness as such, the word "consciousness" is as close as you can get to being able to indicate the phenomenon. So I can't agree that it's not a helpful concept. But in fact, the "thing" that is the referent of that term is not a concept at all. It is the pre-conceptual basis or arena for concepts that arise in it. The TV set is not a TV show. Humans obviously vary in how conscious they are, both from one to another as well as individually over time so it cannot be properly defined circularly as you say above. All "structures and mechanisms of information flow" are contents of consciousness not components. Awareness has no qualities, no form, no sides nor parts, it does not experience time: it is always "now", and it is always "here". What you are seeing "in varying degrees in nature, in computers, and of course in people" is mind I think. Cf. Gregory Bateson, "Mind and Nature: A Necessary Unity" and "Steps to an Ecology of Mind" Lastly, "Godel Escher Bach" is an excellent book and was instrumental in my own process of coming to grips with consciousness. To wit: I think the closest we can come to modelling or describing consciousness mathematically is as a strange loop involving the entire Universe though-out all time.
jcoffland 9y ago
> Now I find it hard to hold on to the belief that I understand what is "A" and what is "B", while computer can only compute. Humans being surprised by the computer should not be the yardstick for AI. A trained neural net can recognize the letter "A" and differentiate it from things that are not "A" but it does not know that "A" is part of the Latin alphabet and that there are other alphabets that form written human languages. The day the computer spontaneously invents a new and usable alphabet without having been specifically designed to do so is the day I will concede we have hard AI. We have a long way to go. Until then it's just a bunch of hotdog/not hotdog classifiers.
- AndrewKemendo 9y ago
  but it does not know that "A" is part of the Latin alphabet and that there are other alphabets that form written human languages. There is nothing preventing the computer for learning those connections however, so all you are doing is moving the abstraction layer. It's not a fundamental break point.
- rdlecler1 9y ago
  I have subsystems in my brain processing he letter A that also do not know that it's part of the Latin Alphabet. It's a start but I agree we have a long long way to go to hard AI and I'd be surprised if I see it in my lifetime.
- electrograv 9y ago
  > The day the computer spontaneously invents a new and usable alphabet without having been specifically designed to do so is the day I will concede we have hard AI. Most humans have not spontaneously invented new usable alphabets, so I suppose that means most humans haven't meet the bar for true intelligence either. I still don't understand this obsession for trying to define "hard AI" or "true intelligence" in binary terms. Intelligence is a spectrum, and deep learning has advanced it forward, thus making machines more intelligent -- yes, we can use that word 'intelligent' for computers just as we do for biological machines. Don't freak out. Is it really so hard to accept that intelligence isn't all-or-nothing?
tomxor 9y ago
Perhaps i'm arguing semantics and this is what the author means but... in your primitive mind, you are able to recognise something even if you have no idea what it is, you can learn to recognise. The ability to introspect and analyse what makes that thing unique or understand what it's purpose or origin is has everything to do with being sentient. We might not know what exactly being sentient is but recognising an image is like lobotomising the brain to just be a visual cortex, it can match but the other networks that work in the abstract are not there.
deleted 9y ago
[deleted]
kumartanmay 9y ago
Isn't human's greatest power in ability to think and imagine. Even animals are conscious and understand their surroundings?
AndrewKemendo 9y ago
Most comments here are to the tune of "Well DL is just a bunch of correlations and statistics, it's not really understanding anything" Ok, well I can also say "humans are just a bunch of chemical reactions and electrical signals." The beauty of DL is in it's simplicity and really we're at the very starting point of seeing it work with extremely sparse networks (compared to biological intelligence). The fact that it works so well with such limited data in narrow domains should be energizing.
- OtterCoder 9y ago
  Enervating? I find it the opposite. It's exciting and energizing to think of what we can do with this.
  - AndrewKemendo 9y ago
    Gah thanks for the catch, gotta love autocorrect.
- carapace 9y ago
  Babbage is said to have owned a dancing automaton he called "The Silver Lady" that was delightfully lifelike in its movements. I wouldn't say that such a device "understood" dance, no matter how perfectly it moved.
  - kthejoker2 9y ago
    Given today's technology and sufficient time, you could devise an AI that could watch dance videos, "understand" dance, and create its own Silver Lady.
  - AndrewKemendo 9y ago
    You're arguing a strawman. I never claimed that understanding was based on a phenomenological evaluation of an output. Rather, reductionism is not an argument against complexity.
    - 9y ago
deafcalculus 9y ago
Consciousness is likely just a whole bunch of computation. I suspect "What is consciousness?" will go the way of "What is life?". We more or less understand things that make up a bacteria. Those components aren't alive although the bacteria is. So, it's just a matter of definition.
- mbrock 9y ago
  What do you mean by computation? What's an example of something that isn't computation?
  - deafcalculus 9y ago
    In this context, I intended it to mean a combination of addition, multiplication, and a small set of relatively simple non-linear functions.
    - RivieraKid 9y ago
      That means you can create consciousness by simulating a Turing machine with pen and paper, or by positioning sand systematically. You can encode its memory in different ways by giving different meaning to different positioning of sand. So randomly throwing sand around could create a Turing computation of consciousness (and all kinds of feelings) with the right choice of encoding.
- RivieraKid 9y ago
  Couldn't disagree more. Consciousness is misunderstood by surprisingly large number of smart people. The common view is that there's science and that's it, when actually, science just describes the patterns of what we observe via consciousness, which is in a way above science. Regarding "what is life?", that's fundamentally different. Life can have fairly concrete definitions. Basically, it's a physical matter with specific properties, that's it. Whereas with consciousness, it's much more complicated. But defining, say, the feeling of pain as a physical matter with specific properties doesn't make much sense. "Pain is when these neurons are charged." Also, what is a computation? A falling rock does perform a computation of a physical process. Any physical system can be said to perform a computation - or even a myriad of different computations, depending on how the physical state is interpreted.
iamleppert 9y ago
He's making the age old mistake of conflating mapping input and outputs to intelligence. Intelligence is not defined by the ability to recognize letters. Or play a game of Go. Deep learning is a powerful tool for creating systems that have an ability to map inputs to outputs with very noisy, non-linear or complex data. The mapping itself may be complex, but it's not going about solving problems like a person would. It has no idea what letters are, and how they fit into its world. It has no concept of self, cannot contemplate its own existence -- and perhaps most important of all, has no free will. The moment we have some kind of deep learning or AI that has free will and can express interest in something other than what it has been trained on, I would say we are closer to unraveling the mystery of consicenesss and human intellect. Even babies are animals exhibit many forms of free will, decision making, and novel behavior that cannot be explained with our current observations of route deep learning techniques.
- likelynew 9y ago
  You are just thinking of very simple scenarios of supervised learning. Even the simplest of other examples like playing games can be thought as decision making(or free will?). Also then there are areas where deep learning research is heading, e.g. neural turing machines. It has just arrived and does not works great, but if the concept will be successful, it can be thought as free will by all definitions.
  - shabeyyub 9y ago
    Isn't free will a non-deterministic thing? NTM, DNC etc. are very promising but at the end they are programs runned by a Turing Machine. Are free will/consciousness computable? this is the real question IMO.
    - AndrewKemendo 9y ago
      Isn't free will a non-deterministic thing? Whoo boy not sure that's a good rabbit hole to go down. If you're unfamiliar with compatibilism I'd suggest you check it out. I think hard determinism gives the most reasonable answer here with a resounding no. As to the question of consciousness, it is yet to be well defined, with no possiblity to test (because of eg Qualia) so by definition you'd never verify or not. At most you'd recognize what you perceive as consciousness based on how you perceive other entities which you believe have it.
freech 9y ago
http://lesswrong.com/lw/iv/the_futility_of_emergence/ http://lesswrong.com/lw/iv/the_futility_of_emergence/

> Computers understand things as well as us, perhaps better. If this was limited to chess, I would unquestionably agree. If it was limited to image recognition, I would tentatively agree, although things like [0] make me cautious (admittedly, that was from March, and I'm not familiar with progress since then). However, the author seems to be generalizing beyond those two domains, to the limits of human understanding. That seems like a couple-orders-of-magnitude leap too far to me. For example, I don't know of any autonomous system capable of understanding a short novel with simple language and writing a one-page summary of it, as might be expected of a human ten-year-old. [0] https://twitter.com/Meaningness/status/846478348947668992 https://twitter.com/Meaningness/status/846478348947668992

amelius 9y ago
Perhaps your reference [0] would work on the human brain too, if only we could know all the weights assigned to all neurons/axons of the given human this should apply to :)
- komaromy 9y ago
  Could be! We'll need a volunteer comfortable with having their neuronal weights experimented on.
  - bitL 9y ago
    You are forgetting local protein-based computations observed in biological neurons we have no clue about...
- statusgraph 9y ago
  Interestingly, you can treat the NN as a black box (ie, not look at individual weights or even the architecture) and still derive adversarial cases: https://arxiv.org/abs/1602.02697 https://arxiv.org/abs/1602.02697
  - amelius 9y ago
    Interesting. Would that work on humans too?
bitL 9y ago
To what extent those universal perturbations are causing problems due to insufficient image augmentation? Or due to deficient optimizer used while training CNNs (all optimizers are just heuristics with nasty failure cases)? Could we train a GAN-like DNN on those perturbations to make their effect disappear?

singham 9y ago

Daniel Dennett has been saying this for quite a while.

bitL 9y ago

I think author is stretching arguments here a bit - DL is just partitioning space according to some pre-baked associations given to it during training; in this case it's more like a non-linear optimization where we want to end up with N-million dimensional objects of certain shape obtained by optimizing some objective function allowing predicting similar associations. It doesn't have much with the actual innate quality of understanding. Maybe reinforcement learning with deep learning together (DRL) can move us towards such a quality at least in a mechanical sense.

petters 9y ago
On one hand, I sort of agree with you. On the other hand, what you are saying feels a little bit like saying that humans aren't impressive, because we are just atoms. Sometimes interesting things arise from many small, simple parts.
jacquesm 9y ago
That's got to be one of the most concise but still complete descriptions of deep learning that I've seen so far. The question that it implicitly raises (at least, with me) is how can we tell the difference between 'understanding' and 'deep learning' if the end results are the same? To me 'reasoning' is a slow, conscious process, and understanding is a part of that. But classification problems , especially when done by humans when they try to work fast have no room for such conscious decision making, we go much faster than that and outsource the job to our subconscious. Predictably, the error rate goes up and in those kind of situations deep learning can today already outperform humans on the same tasks. The weird thing is that deep learning solutions can get simple cases completely wrong, where a human would never err, and yet get some of the hardest cases - where a human would be very likely to make an error - right. It's baffling.
akyu 9y ago
>It doesn't have much with the actual innate quality of understanding That was his entire point.
westoncb 9y ago
> DL is just partitioning space according to some pre-baked associations given to it during training What I wonder is whether that's not also maybe the cornerstone of human understanding. If I understand correctly, you are essentially saying that DL is forming categories, or developing a classification scheme. Granted, if we're only talking about supervised DL, and the program is practically told where to form the boundaries—then it's not very impressive. But if the software is extracting statistically prominent commonalities and using those to form category boundaries, and arranging them hierarchically—then while the implementation may be totally different from human understanding, the effect seems to strongly overlap. I assume I'm probably just missing something—anyone know what it is? (It seems clear that at least part of the problem here is that 'human understanding' has been left far more vague than DL, and in order to say one way or the other how much they have in common, we need to better define 'human understanding'.)

mathgenius 9y ago

I don't see why "understanding" is equivalent to mere pattern recognition. Even using this word "recognition", what does that mean? It's another word like "understand". These algorithms are just pattern patterning. They don't even know they are patterning, that is a meta-property assigned in (or by) a context.

ffwd 9y ago
I agree, but I think human knowledge can be represented as either a graph, hierarchy or network of patterns. Like my knowledge of the letter 'A' is a network of connections to patterns 'language', 'english', 'alphabet', whatever else, and if the computer can do the same, it can use that knowledge of that network (as a whole separate entity) to make a decision, so to speak. Consciousness does come into it since we have a pretty visceral sense of it, and especially when we mentally trawl through our patterns to make some story, but really understanding should just be creating new patterns from existing patterns and the ability to utilize them as distinct entities in some way (rather than being emergent in the system implicitly and only being utilized by accident, say as emergent behavior randomly occurring because of local constraints)
AndrewKemendo 9y ago
I don't see why "understanding" is equivalent to mere pattern recognition. You're underestimating what goes into high accuracy pattern recognition as well as assuming that patterns exist for only one vector and in a single context. If I asked you to explain how you "understand" some concept, it will inevitably be how the structure and mechanics of it relate to others and in what context. All of those are simply patterns that are abstracted or made more granular. For example, how do you "understand" what a car is? You would inevitably describe some definition of a car mechanically and the context in which a car operates. So it's a contained combination of metal and plastic objects and usually liquids with a mechanism to transfer power through gearing and wheels, a compartment for humans, some control mechanisms etc... (definition of the technical), but it can't operate in water (boat) or in the air (airplane). Each of these things is learned through exposure over time, and recognized as connected, to come up with a "understanding" of a car even before it's formally defined. This is why children ask if cars can fly or go in the water.

AndrewOMartin 9y ago

Searle's Chinese Room Argument was specifically aimed at people claiming an algorithm could understand something because of its behaviour. It applies to Deep Learning as much as it does Schank and Ableson's script understanding system.

nightski 9y ago
If a human can translate perfectly without understanding the conversation, then that to me implies that the mind itself gives no innate intelligence similar to the computer. It must be taught the meaning of things, exactly as a computer would need to be. I'm just not following his logic, it feels like a straw man. Of course the computer doesn't understand the meaning of the symbols it is translating, because it was never given data to teach it that (similar to a human in the scenario).
inventtheday 9y ago
The Chinese Room Argument is deeply flawed because it assumes that language translation in humans is a conscious phenomenon. In fact, if you're proficient in a foreign language, you can relate to the fact that for the most part translation happens in the black box of the subconscious mind. The words "bubble out" naturally. The black box of the subconscious mind is no different than the black box of the Chinese room. "Understanding" in the traditional sense is absent from both processes.

dna_polymerase 9y ago

> Given enough examples, computers can understand what is letter "A" and what is letter "B". Meh. Given enough examples, computers now can distinguish letter A and B but distinguishing is not understanding. You could argue that after learning the Network just uses an instruction set and from the outside that may leave the impression of understanding but it really does not. Isn't that basically the Chinese room thing?

hyperbovine 9y ago
In fact recent research indicates that you can randomly relabel the training examples and the network still achieves zero training error (https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530). So it is not "understanding" anything intrinsic or fundamental about the letter "A". Rather, it's just storing training examples somewhere inside of its millions of parameters, which sounds a lot less impressive.
- jimfleming 9y ago
  That is not a conclusion that can be drawn from the findings in the paper. While the models they evaluate can achieve zero training error on random labels, the test error is obviously not zero: it doesn't generalize at all. However, training on real labels often finds solutions which can generalize quite well. A better way to summarize the central question of this paper would be: "Why is it that a large-parameter model trained with gradient descent on real data _could_ just memorize all of the training data (it has the capacity) yet finds solutions which generalize well to an unseen test set?" To say that deep learning is _just_ memorizing its training data would be incorrect. We have empirical evidence to the contrary and this paper is part of that evidence.
  - hyperbovine 9y ago
    But we also have empirical evidence that they generalize incredible poorly, namely the existence of imperceptible (adversarial) perturbations which can transfer across images and networks and are catastrophically misclassified.

jeremynixon 9y ago

The mysticism around ‘Emergence’ is just a modeling error where people only abstract in one way (say, down to cells) and don’t include something important like the interaction between cells in their reductionist model of the system. It’s like creating a graph without the edges. And so when those effects have manifest consequences at a higher level, it feels like they appeared as if by magic.

ktRolster 9y ago
IT's kind of like saying, "This river is not the same river that it was upstream, and yet it is. The river is not the same water of last year, and yet it is the same river." The phenomenon is entirely well understood by all involved, and yet coming up with a reasonable definition is hard. http://existentialcomics.com/comic/164 http://existentialcomics.com/comic/164 So it's easier to be mystical.

mannykannot 9y ago

I have always believed that understanding is an emergent property of physical processes that could be modeled computationally, but I do not think deep learning has yet demonstrated that it has yet achieved it. Some of the evidence comes from the ways it fails, such as 'recognizing' images that humans would understand are not what the systems think they are, and being confident in decisions that make no sense. These situations occur precisely because of a lack of understanding. I am open to the possibility that deep learning alone might achieve understanding, but I think it is more likely to succumb to the law of diminishing returns before it gets there.

inventtheday 9y ago

Actually, computers are conscious as well. Consciousness is simply a system of information that operates on a continuous sense/plan/act loop. You could argue that they are "less" conscious, but to say that they are unconscious is to make the same mistake as people have made for years by saying that computers cannot "understand" anything. Some people push back on this by saying computers have no sense of self. Thats not true. Most computers do have internal state representations about themselves. Take a driverless car for example. When it does localization, it's constantly referencing its own shape and speed and comparing it to the environment. That's a sense of self. Whatever philosophical barriers we place between ourselves and machines (and animals/nature for that matter), one thing is for certain: they will eventually debunked.

gfodor 9y ago
Your last claim may be true, but your assertions don't constitute any form of debunking. We don't have an objectively agreed upon way to measure consciousness (though some have been proposed) so making bold claims like "computers are conscious as well" doesn't make much sense until we agree on a way to measure and experiment on the presence of consciousness.
- inventtheday 9y ago
  Society has assumed the defacto circular definition of consciousness as "whatever we, as humans, are experiencing". For obvious reasons, this is not a helpful concept. Instead, I like to think of consciousness in terms of structures and mechanisms of information flow. If we open our minds up to this type of thinking, we can see consciousness in varying degrees in nature, in computers, and of course in people. For anyone who's curious, the guiding light in this school of thought is Hofstadter's Godel Escher Bach.
  - carapace 9y ago
    I upvoted your original comment above even though I don't agree because you're commenting in good faith in my opinion. Society as a whole has not assumed a concrete definition for consciousness, as there are a lot of people who don't give it a second thought. Among those who do recognize consciousness as such, the word "consciousness" is as close as you can get to being able to indicate the phenomenon. So I can't agree that it's not a helpful concept. But in fact, the "thing" that is the referent of that term is not a concept at all. It is the pre-conceptual basis or arena for concepts that arise in it. The TV set is not a TV show. Humans obviously vary in how conscious they are, both from one to another as well as individually over time so it cannot be properly defined circularly as you say above. All "structures and mechanisms of information flow" are contents of consciousness not components. Awareness has no qualities, no form, no sides nor parts, it does not experience time: it is always "now", and it is always "here". What you are seeing "in varying degrees in nature, in computers, and of course in people" is mind I think. Cf. Gregory Bateson, "Mind and Nature: A Necessary Unity" and "Steps to an Ecology of Mind" Lastly, "Godel Escher Bach" is an excellent book and was instrumental in my own process of coming to grips with consciousness. To wit: I think the closest we can come to modelling or describing consciousness mathematically is as a strange loop involving the entire Universe though-out all time.

jcoffland 9y ago

> Now I find it hard to hold on to the belief that I understand what is "A" and what is "B", while computer can only compute. Humans being surprised by the computer should not be the yardstick for AI. A trained neural net can recognize the letter "A" and differentiate it from things that are not "A" but it does not know that "A" is part of the Latin alphabet and that there are other alphabets that form written human languages. The day the computer spontaneously invents a new and usable alphabet without having been specifically designed to do so is the day I will concede we have hard AI. We have a long way to go. Until then it's just a bunch of hotdog/not hotdog classifiers.

AndrewKemendo 9y ago
but it does not know that "A" is part of the Latin alphabet and that there are other alphabets that form written human languages. There is nothing preventing the computer for learning those connections however, so all you are doing is moving the abstraction layer. It's not a fundamental break point.
rdlecler1 9y ago
I have subsystems in my brain processing he letter A that also do not know that it's part of the Latin Alphabet. It's a start but I agree we have a long long way to go to hard AI and I'd be surprised if I see it in my lifetime.
electrograv 9y ago
> The day the computer spontaneously invents a new and usable alphabet without having been specifically designed to do so is the day I will concede we have hard AI. Most humans have not spontaneously invented new usable alphabets, so I suppose that means most humans haven't meet the bar for true intelligence either. I still don't understand this obsession for trying to define "hard AI" or "true intelligence" in binary terms. Intelligence is a spectrum, and deep learning has advanced it forward, thus making machines more intelligent -- yes, we can use that word 'intelligent' for computers just as we do for biological machines. Don't freak out. Is it really so hard to accept that intelligence isn't all-or-nothing?

tomxor 9y ago

Perhaps i'm arguing semantics and this is what the author means but... in your primitive mind, you are able to recognise something even if you have no idea what it is, you can learn to recognise. The ability to introspect and analyse what makes that thing unique or understand what it's purpose or origin is has everything to do with being sentient. We might not know what exactly being sentient is but recognising an image is like lobotomising the brain to just be a visual cortex, it can match but the other networks that work in the abstract are not there.

deleted 9y ago

[deleted]

kumartanmay 9y ago

Isn't human's greatest power in ability to think and imagine. Even animals are conscious and understand their surroundings?

AndrewKemendo 9y ago

Most comments here are to the tune of "Well DL is just a bunch of correlations and statistics, it's not really understanding anything" Ok, well I can also say "humans are just a bunch of chemical reactions and electrical signals." The beauty of DL is in it's simplicity and really we're at the very starting point of seeing it work with extremely sparse networks (compared to biological intelligence). The fact that it works so well with such limited data in narrow domains should be energizing.

OtterCoder 9y ago
Enervating? I find it the opposite. It's exciting and energizing to think of what we can do with this.
- AndrewKemendo 9y ago
  Gah thanks for the catch, gotta love autocorrect.
carapace 9y ago
Babbage is said to have owned a dancing automaton he called "The Silver Lady" that was delightfully lifelike in its movements. I wouldn't say that such a device "understood" dance, no matter how perfectly it moved.
- kthejoker2 9y ago
  Given today's technology and sufficient time, you could devise an AI that could watch dance videos, "understand" dance, and create its own Silver Lady.
- AndrewKemendo 9y ago
  You're arguing a strawman. I never claimed that understanding was based on a phenomenological evaluation of an output. Rather, reductionism is not an argument against complexity.
  - 9y ago

deafcalculus 9y ago

Consciousness is likely just a whole bunch of computation. I suspect "What is consciousness?" will go the way of "What is life?". We more or less understand things that make up a bacteria. Those components aren't alive although the bacteria is. So, it's just a matter of definition.

mbrock 9y ago
What do you mean by computation? What's an example of something that isn't computation?
- deafcalculus 9y ago
  In this context, I intended it to mean a combination of addition, multiplication, and a small set of relatively simple non-linear functions.
  - RivieraKid 9y ago
    That means you can create consciousness by simulating a Turing machine with pen and paper, or by positioning sand systematically. You can encode its memory in different ways by giving different meaning to different positioning of sand. So randomly throwing sand around could create a Turing computation of consciousness (and all kinds of feelings) with the right choice of encoding.
RivieraKid 9y ago
Couldn't disagree more. Consciousness is misunderstood by surprisingly large number of smart people. The common view is that there's science and that's it, when actually, science just describes the patterns of what we observe via consciousness, which is in a way above science. Regarding "what is life?", that's fundamentally different. Life can have fairly concrete definitions. Basically, it's a physical matter with specific properties, that's it. Whereas with consciousness, it's much more complicated. But defining, say, the feeling of pain as a physical matter with specific properties doesn't make much sense. "Pain is when these neurons are charged." Also, what is a computation? A falling rock does perform a computation of a physical process. Any physical system can be said to perform a computation - or even a myriad of different computations, depending on how the physical state is interpreted.

iamleppert 9y ago

He's making the age old mistake of conflating mapping input and outputs to intelligence. Intelligence is not defined by the ability to recognize letters. Or play a game of Go. Deep learning is a powerful tool for creating systems that have an ability to map inputs to outputs with very noisy, non-linear or complex data. The mapping itself may be complex, but it's not going about solving problems like a person would. It has no idea what letters are, and how they fit into its world. It has no concept of self, cannot contemplate its own existence -- and perhaps most important of all, has no free will. The moment we have some kind of deep learning or AI that has free will and can express interest in something other than what it has been trained on, I would say we are closer to unraveling the mystery of consicenesss and human intellect. Even babies are animals exhibit many forms of free will, decision making, and novel behavior that cannot be explained with our current observations of route deep learning techniques.

likelynew 9y ago
You are just thinking of very simple scenarios of supervised learning. Even the simplest of other examples like playing games can be thought as decision making(or free will?). Also then there are areas where deep learning research is heading, e.g. neural turing machines. It has just arrived and does not works great, but if the concept will be successful, it can be thought as free will by all definitions.
- shabeyyub 9y ago
  Isn't free will a non-deterministic thing? NTM, DNC etc. are very promising but at the end they are programs runned by a Turing Machine. Are free will/consciousness computable? this is the real question IMO.
  - AndrewKemendo 9y ago
    Isn't free will a non-deterministic thing? Whoo boy not sure that's a good rabbit hole to go down. If you're unfamiliar with compatibilism I'd suggest you check it out. I think hard determinism gives the most reasonable answer here with a resounding no. As to the question of consciousness, it is yet to be well defined, with no possiblity to test (because of eg Qualia) so by definition you'd never verify or not. At most you'd recognize what you perceive as consciousness based on how you perceive other entities which you believe have it.

freech 9y ago

http://lesswrong.com/lw/iv/the_futility_of_emergence/ http://lesswrong.com/lw/iv/the_futility_of_emergence/