9 ms·

He's right about LeCun. Yann really does not like Deep Learning being criticized. Please someone who knows much more than me about deep learning tell me how a

• ben_w 8y ago
  > Please someone who knows much more than me about deep learning tell me how a deep learning ai can explain how it came to an answer. > It can't. By definition. What definition are you using? From the my understanding, if you can explain your reasoning, then a deep learning system can in principle learn how to explain its reasoning. If it couldn’t, you couldn’t either.
  • opless 8y ago
    We're not deep learning systems. So reasoning by analogy there isn't correct. (Aside: There have been experiments that show that many explanation for [everyday] actions are, in fact made up after the decision to execute an action. Which gives some insight to the whole nature of free will - but that's another discussion entirely. However, indeed we're able to show our workings for higher level workings, like mathematical proofs. ) Deep learning systems, are examples of supervised learning. Once trained, they cannot adapt to new input. You have to retrain to accommodate new features that you want to capture. Deep learning systems, are in essence a very long equation that hides all meaning over the entire network - good luck in finding out why it came to a decision. You might get lucky and see a definitive triggering path through the network, but that's an exception to the rule, as far as I understand. (I'm willing to be proven wrong here.) Symbolic systems, can trivially be made to trace the code execution paths they made for any action they might take. (also - hi Ben! I recognise your blog!)
    • FakeComments 8y ago
      I don’t think we’re able to show our workings for higher level things like proofs. For instance, I can’t tell you (accurately) how I composed the proof conceptually, ie what strategy it takes, or even determined the appropriate steps, ie how I determined to use a particular tactic. I can tell you the premises lead to the conclusion via a chain of reasoning, but that’s the artifact of my thought process, not anything about how it works. The output of proving is that chain of steps. So I think here is somewhere DL systems may have an advantage: they actually can freely introspect their thought process as part of their thought process.
      • red75prime 8y ago
        > they actually can freely introspect their thought process as part of their thought process. Do you mean that they can in principle? I don't know existing systems, which can inspect their own weights and output a vector of confidence scores that they can recognize classes A, B, C, ...
        • deleted 8y ago
          [deleted]
        • p1esk 8y ago
          output a vector of confidence scores that they can recognize classes A, B, C, ... This is literally what neural networks do when classifying patterns.
          • red75prime 8y ago
            They classify patterns, not their own ability to classify patterns.
            • p1esk 8y ago
              Again, they literally produce confidence scores - probabilities that each prediction is correct. For example, say there are 3 classes, and the network is shown two different examples of class 2. Say it outputs a class probability distribution {0.02, 0.45, 0.43} for the first example, and {0.02, 0.9, 0.08} for the second. Even though in both cases it correctly identifies class 2, it's a lot more confident in its prediction in the second case.
  • Symmetry 8y ago
    Humans can almost always give an explanation judgement but, for intuitive judgements at least, the reasons we give are frequently wrong or incomplete. People seldom notice that we rate essayists with more symmetric faces as better at writing but we have this bias and tons more besides. But I don't think that there's any reason to doubt that when using abstract reasoning, Kahneman's system 2, our explanations of our thought processes can't be accurate. It's system 1 that's opaque to accurate introspection.
    • szemet 8y ago
      Aren't we - as a group - better in this? I'm thinking about the scientific method, and the common differentiation there between the context of discovery and the context of justification. So you are free to discover a mathematical theorem by intuition or during daydreaming (like the molecular structure of benzene) - anything goes. But in the context of justification, you have to be much more disciplined - even if you trust your own intuitions and don't need to persuade yourself (of course generally you should not trust your intuition in scientific context that much;), you have to persuade others who are missing your insight. edit: And of course - in the end - to accept something as a true A.I. system, it probably has to persuade us that its intuitions are right, using the type justifications we accept. (So even if it has better types of justification - if we won't get it, they has to dumb it down to our level of scientific understanding)
      • red75prime 8y ago
        We are verifying the results of our opaque brain processes. We cannot yet verify that our explanations of those processes are correct.
    • Nasrudith 8y ago
      Given some results regarding brain damage we might just be rationalizing afterwards separate from the problem solving. Severed corpus callosum I believe it was. Or that effect might just be how a brain keeps functioning after losing some pieces - better off evolutionarily have data integrity issues than shut down if a checksum fails. I believe that ML can technically give you an answer but it ammounts to a summation of its training set and how it parses the data. Which isn't too readable to even mathematicians and leaves it vulnerable to stuff like adding a small pixel cloud to cause it to see a toy turtle as an assault rifle. Our errors certainly exist for visual processing but they tend to be more distorion than that.
• kahoon 8y ago
  > Please someone who knows much more than me about deep learning tell me how a deep learning ai can explain how it came to an answer. A mouse can't do that either. Yet we are unable to mimic mouse intelligence with any of our current tools.
• empath75 8y ago
  Most people can’t explain why they make most of their decisions, really. It’s really only a very small subset of our activity that we are capable of explaining logically. What’s the chain of reasoning behind me falling in love and marrying my wife? What’s the chain of reasoning behind why I watch the great British bake off compulsively? What’s the chain of reasoning that leads an alcoholic to drink time and time again. What’s the chain of reasoning behind why I look at a rainbow and see bands of distinct colors? What’s the chain of reasoning behind why I paused and scratched my nose just now?
• marcosdumay 8y ago
  A trained neural network is a model, just like any other kind of model. If there is any overall property of the data it learned, you can coerce it from the network just like you can read them from any mathematical model. (What is, for some properties, very easily, and for others it's very hard.) An AI can always explain how it came to an answer. But the explanation can be like "there are way too many factors" or "things are like this just because of those other things, if those changed, the explanation would too". But those are not an attribute of the neural networks, but of the problem they are solving. If you get an explanation like that for a problem that has simple explanations, it means you failed at training your network.