3 ms·

>> I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of cou

by dmreedy 10y ago

>> I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of course...) and would prefer that their preferred methods would still reign supreme. I think that's definitely a part of it, and I feel that way sometimes myself (not that I'm a PhD). But there's another side of that reluctance that lies on the axis of model accountability and explicability. A lot of modern ML/Deep Learning doesn't -feel- like we're understanding anything any more than we did ten years ago. Yes our black-box results are better according to the tests we've laid out for them, but there's something more slippery about the 'why', beyond the handwave of 'complexity'. Maybe this is just the way it will be going forward (in the spirit of Quantum's "shut up and calculate"), but it is not easy to give up something that you can wrap your head around with something that kind of just takes care of itself, especially if you're in the business of seeking knowledge instead of results.

deepnotderp 10y ago
I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either. On the other hand, we're nowhere near human level intelligence in most tasks (Go, Poker and image classificaiton non withholding) so I can understand the argument for explainability from a practical perspective. I think we're making some good progress in that direction though, and I'll list them below: 1) Attention maps in CNNs can tell us what the net is usually looking at. 2) "Attentive Explanations" use attention mechanisms to point to the object of interest to generate an explanation for VQA tasks, check the paper (warning PDF): https://arxiv.org/pdf/1612.04757 https://arxiv.org/pdf/1612.04757 3) A recent project used a similar explanation mechanism that forced the network to output "what it was thinking" while playing an Atari game. 4) NTMs (neural turing machines) allow weighted memory access, which alleviates the back box issue to some extent.
- dmreedy 10y ago
  >> since our goal is human level intelligence and human intelligence isn't easily explained either I think this might be where the real divide lies. To present a false dichotomy, there are two schools. One that is interested in building systems that attain human-level intelligence, and another that is interested in understanding the nature of human intelligence by building models using our closes understood analog (computation). We're definitely making progress on Deep Learning models that tell us more about how they think they work, but philosophically, I'm not sure we're making progress on understanding the nature of what they experience. I don't mean this in any polemical sense, just trying to explain the particular dissatisfaction (and it -is- dissatisfaction, not contempt or disdain. The stuff is amazing) I feel towards these advances.
- romaniv 10y ago
  >I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either. There are two types of explainability. One is explaining why the system converges on correct solutions in general. Another is explaining why a particular decision was made the way it was made. Human intelligence is usually very much explainable in the second sense. This is not always true, but such cases are called "intuition" and we tends not to trust such judgements for mission-critical tasks.
- ylem 10y ago
  The following article has an interesting treatment of interpretability: https://arxiv.org/pdf/1606.03490.pdf https://arxiv.org/pdf/1606.03490.pdf
- sgt101 10y ago
  Explainability is important - understanding why and how, but when it comes to the what the issue is accountability. If you are dragged up in court and accused of manslaughter (hello 5 years in an open prison) then you really want to be able to say, hand on heart, that what you did was professional and that you could reasonably have expected "not to kill". "I A/B tested it and everything checked out" isn't going to cut it. Also; the nights in the prison cell contemplating that you are held responsible for offing some poor sod. Not so good.
sgt101 10y ago
Yup, can't explain it, don't understand it.. let's let it loose on pediatric surgery what can go wrong!
cr0sh 10y ago
I understand and sympathize with your thoughts; I'm not an ML or deep learning expert, but I do agree somewhat with Mark Cuban's assertion that software developers need to get familiar with the technology. I'm currently enrolled in the Udacity Self-Driving Car Engineer Nanodegree, and while that program is focused on applying ML techniques to a specific problem domain, I can see where and how to apply what we've learned so far to other problem domains. Furthermore, I have in the past taken other MOOC courses (Ng's ML Class in 2011, and Udacity's CS373 in 2012) which taught more generalized approaches (more of the "standard" ML concepts in Ng's course, less so in the CS373 course - but I should note that even in the course I am taking currently, one or two of the lessons did discuss and talk about GD and other simple mathematical models for ML for classification, etc). I recently completed (late) the "behavioral cloning" project, where we had to implement a CNN to drive a simulated car around a track. At first, it seemed like it would be a simple challenge - slap the NVIDIA model on the thing, gather a bunch of data for training/validation, and it should all "just work". If only. I struggled immensely with that one; when trying to explain the problem to "outsiders", I kept running into the issue (which I knew about) of how to describe the fact that this system wasn't something I could easily explain. I can't tell someone "oh change that rule and tweak that test and it does this" - because it doesn't. Its not that simple. I know this, but conveying it to someone else isn't easy. I've found that these systems, for their simplicity compared to, say, a mouse's brain - share the same kind of "black box" issues - I can't tell you why my car likes to veer at a certain point, then recover. I never drove like that with my training data, but for some reason, somehow - that's what it learned. From a business perspective - this is where things will fall down. We (that is, those people in the deep learning ML domain space) can't explain how these models work. In some ways, they don't seem to be deterministic. In theory, post training, they should be - but in reality, short of perfect data being fed in - the data they get in their inputs is never exactly the same - their ability to generalize and fit this imperfect data, with a result that correct, within a level of error - is a strength, not a weakness. But try to convey this to someone where money or exact results are on the line... Researchers can't even explain how the C.Elegans connectome works, and it is vastly simpler and completely mapped (interestingly, embodying the connectome as an ANN in a robot produces a machine that acts and can be trained like a C.Elegans - we don't how or why, though). We humans expect determinism from our machines, because for so long that is how machines have operated; the same input produces the same output. Our machines now, though, can produce different outputs for the (seemingly) same input, and that worries us (perhaps rightfully so, especially in the case of a self-driving anything carrying passengers!). Interestingly, we give a "pass" to human beings doing these same tasks; that is, we don't expect perfect results from these non-deterministic neural networks we hire or employ. We don't worry (much - though we try with imperfect models) about how one person decides to do something one way or another, even if they can't explain themselves. But we create machines that do that same - and it worries us. I sometimes wonder if that says more about us than about the machines...

by dmreedy 10y ago

deepnotderp 10y ago

I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either. On the other hand, we're nowhere near human level intelligence in most tasks (Go, Poker and image classificaiton non withholding) so I can understand the argument for explainability from a practical perspective. I think we're making some good progress in that direction though, and I'll list them below: 1) Attention maps in CNNs can tell us what the net is usually looking at. 2) "Attentive Explanations" use attention mechanisms to point to the object of interest to generate an explanation for VQA tasks, check the paper (warning PDF): https://arxiv.org/pdf/1612.04757 https://arxiv.org/pdf/1612.04757 3) A recent project used a similar explanation mechanism that forced the network to output "what it was thinking" while playing an Atari game. 4) NTMs (neural turing machines) allow weighted memory access, which alleviates the back box issue to some extent.

dmreedy 10y ago
>> since our goal is human level intelligence and human intelligence isn't easily explained either I think this might be where the real divide lies. To present a false dichotomy, there are two schools. One that is interested in building systems that attain human-level intelligence, and another that is interested in understanding the nature of human intelligence by building models using our closes understood analog (computation). We're definitely making progress on Deep Learning models that tell us more about how they think they work, but philosophically, I'm not sure we're making progress on understanding the nature of what they experience. I don't mean this in any polemical sense, just trying to explain the particular dissatisfaction (and it -is- dissatisfaction, not contempt or disdain. The stuff is amazing) I feel towards these advances.
romaniv 10y ago
>I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either. There are two types of explainability. One is explaining why the system converges on correct solutions in general. Another is explaining why a particular decision was made the way it was made. Human intelligence is usually very much explainable in the second sense. This is not always true, but such cases are called "intuition" and we tends not to trust such judgements for mission-critical tasks.
ylem 10y ago
The following article has an interesting treatment of interpretability: https://arxiv.org/pdf/1606.03490.pdf https://arxiv.org/pdf/1606.03490.pdf
sgt101 10y ago
Explainability is important - understanding why and how, but when it comes to the what the issue is accountability. If you are dragged up in court and accused of manslaughter (hello 5 years in an open prison) then you really want to be able to say, hand on heart, that what you did was professional and that you could reasonably have expected "not to kill". "I A/B tested it and everything checked out" isn't going to cut it. Also; the nights in the prison cell contemplating that you are held responsible for offing some poor sod. Not so good.

sgt101 10y ago

Yup, can't explain it, don't understand it.. let's let it loose on pediatric surgery what can go wrong!

cr0sh 10y ago

I understand and sympathize with your thoughts; I'm not an ML or deep learning expert, but I do agree somewhat with Mark Cuban's assertion that software developers need to get familiar with the technology. I'm currently enrolled in the Udacity Self-Driving Car Engineer Nanodegree, and while that program is focused on applying ML techniques to a specific problem domain, I can see where and how to apply what we've learned so far to other problem domains. Furthermore, I have in the past taken other MOOC courses (Ng's ML Class in 2011, and Udacity's CS373 in 2012) which taught more generalized approaches (more of the "standard" ML concepts in Ng's course, less so in the CS373 course - but I should note that even in the course I am taking currently, one or two of the lessons did discuss and talk about GD and other simple mathematical models for ML for classification, etc). I recently completed (late) the "behavioral cloning" project, where we had to implement a CNN to drive a simulated car around a track. At first, it seemed like it would be a simple challenge - slap the NVIDIA model on the thing, gather a bunch of data for training/validation, and it should all "just work". If only. I struggled immensely with that one; when trying to explain the problem to "outsiders", I kept running into the issue (which I knew about) of how to describe the fact that this system wasn't something I could easily explain. I can't tell someone "oh change that rule and tweak that test and it does this" - because it doesn't. Its not that simple. I know this, but conveying it to someone else isn't easy. I've found that these systems, for their simplicity compared to, say, a mouse's brain - share the same kind of "black box" issues - I can't tell you why my car likes to veer at a certain point, then recover. I never drove like that with my training data, but for some reason, somehow - that's what it learned. From a business perspective - this is where things will fall down. We (that is, those people in the deep learning ML domain space) can't explain how these models work. In some ways, they don't seem to be deterministic. In theory, post training, they should be - but in reality, short of perfect data being fed in - the data they get in their inputs is never exactly the same - their ability to generalize and fit this imperfect data, with a result that correct, within a level of error - is a strength, not a weakness. But try to convey this to someone where money or exact results are on the line... Researchers can't even explain how the C.Elegans connectome works, and it is vastly simpler and completely mapped (interestingly, embodying the connectome as an ANN in a robot produces a machine that acts and can be trained like a C.Elegans - we don't how or why, though). We humans expect determinism from our machines, because for so long that is how machines have operated; the same input produces the same output. Our machines now, though, can produce different outputs for the (seemingly) same input, and that worries us (perhaps rightfully so, especially in the case of a self-driving anything carrying passengers!). Interestingly, we give a "pass" to human beings doing these same tasks; that is, we don't expect perfect results from these non-deterministic neural networks we hire or employ. We don't worry (much - though we try with imperfect models) about how one person decides to do something one way or another, even if they can't explain themselves. But we create machines that do that same - and it worries us. I sometimes wonder if that says more about us than about the machines...