4 ms·
If deep learning researchers had a strong theoretical understanding of their own field, a "grand unified theory of deep learning" (and possibly all learning) th
by YeGoblynQueenne 9y ago
If deep learning researchers had a strong theoretical understanding of their
own field, a "grand unified theory of deep learning" (and possibly all
learning) then they wouldn't need special tricks to do explanation .
Unfortunately, the deep learning field suffers from what John McCarthy used to
call the "Look ma, no hands" approach to AI: that's when you get a computer to
do something that hasn't beeen done before with a computer and publish a paper
to announce it without any attempt to identify and study any intellectual
mechanisms.
So the majority of deep learning papers are result papers: someone tweaks an
existing architecture, or invents a new one, to do something new, or beat the
state-of-the-art results. Theoretical papers are very few and far between and
often come from outside the field (like the Renormalisation Group paper, or
the papers on Information Bottleneck Theory).
I don't see how work like the one described in the article bucks the trend.
Visualisation may be intuitive, but any two people can see completely
different things in the same image, especially complex images of large scale
activations. The result is an interpretation method that's up to, well,
personal interpretation. What's more- this only works with vision, where
activations can sort of map to images. It's no use for, say, text, sound, or
other types of data (despite what the article says).
Articles like this tell me that deep learning researchers have basically given
up on understanding how their own stuff works in the course of their careers
and simply accept that their success must come from beating benchmarks and
producing pretty graphs.
A pitty.
- colah3 9y agoI'd like to offer a more optimistic counter view. I think it's likely that deep learning has stumbled upon something deep and profound. And now we're at the point of struggling to make sense of it. Of course things are messy: we're knee deep in the business of trying to start to sort things out. In the optimistic view, the ideas we're grappling with -- ideas like feature visualization, attribution, etc -- might be the seeds of deep abstractions like calculus or information theory. (Of course, these early versions are messy! For example, early calculus was deeply criticized by figures like Berkeley, and took more than a century to put on firm footing via the introduction of limits.) Powerful, novel abstractions may not look like what you expect at first. I do think it's very reasonable of you to be skeptical, of course. Most attempts to craft new ways of thinking about hard problems don't pan out. But I think it's worth pushing really hard on them, because if they do they're very valuable. I feel like we have initial promising results -- give us a decade to see where they go! :) But we could also be totally barking up the wrong tree. :) > What's more- this only works with vision, where activations can sort of map to images. It's no use for, say, text, sound, or other types of data (despite what the article says). That's a reasonable concern. We didn't give any demonstrations of our methods outside vision in the article. I can say that we have done very early-stage prototypes that suggestion similar interfaces work in other domains. Of course, instead of images you get symbols in what ever domain you're working with -- such as audio or text.
- cosmic_ape 9y agoCould you point to some reasons why you'd think that there is anything deep and profound about deep learning? As far as the evidence I know goes, its very shallow as a scientific concept. All there is, is the idea of using convolutions for image feature extraction, which predates "deep learning" by decades. And there is little evidence that the nets do anything more than memorization, which is hardly profound too.
- YeGoblynQueenne 9y agoThank you for taking the time to write a substantial reply! You might be right about deep learning having stumbled upon something deep and profound. Or it may just be the case of "big machine performs well at task that is hard for humans". Like you say, we will have to wait and see. It's just that we won't be seeing much, unless the field focuses real effort on the task of coming up with some kind of "calculus of (deep) learning". As things go right now, it might take more than ten years to see the progress you're hoping for. On a personal note, I should say that I do quite like your idea, in principle. But that's because you're proposing a grammar of design spaces; I think we should use grammars everywhere :P On the demonstration of your technique in other domains than vision- well, that would be really interesting to see. I watched a presentation by a gentleman called Willem Zuidema recently, whose work is on computational linguistics. His team had worked to interpret their deep learning models by visualising their hidden unit activations; he said that it was extremely painful and didn't scale well (he was talking at CoCoSym 2018, a workshop that featured much work on the prospects of combining deep learning with symbolic techniques, mainly for interpretation). If your method can work well in other domains it will definitely be useful to many people. It's still not the kind of theoretical result I'm hoping for, but it would be nice to see a principled way to extract symbolic representations from a continuous space - if that's what you're talking about. Anyway, I'll keep an eye out :)
- cs702 9y ago> Articles like this tell me that deep learning researchers have basically given up on understanding how their own stuff works in the course of their careers and simply accept that their success must come from beating benchmarks and producing pretty graphs. ...a pity This is an overgeneralization, and not a fair one, in my view. The real issue you're bringing up, as I see it, is that we have yet to come up with a set of mental tools necessary for reasoning about deep learning in a rigorous way. In other words, I believe our position is analogous to that of, say, educated people in the early 17th century who could not reason rigorously about Zeno's "paradoxes of motion" because the ideas of calculus had not yet been proposed by Newton and Leibniz. A considerable amount of human intelligence and ingenuity goes into the design and development of novel deep learning systems. I speak from experience. Every tweak and design decision in every model has a reason for being. These deep learning models are not good at what they do by accident!
- YeGoblynQueenne 9y agoYou summarise my comment very well; I'm sorry it feels unfair to you but I hope you can take it as an example of the frustration of outsiders looking in and I think I make it clear why I feel that way. I'm also pretty sure you're right and that the performance of deep learning models is not down to pure chance, but that intuition does us no good while we can't say why they work well. I understand that a fair bit of thought goes into deep learning models- but what kind of thought is that? I get the feeling that most deep learning papers simply describe the most successful of a series of knob-turning experiments. "We turned this knob here and performance went up by 0.05%!". I can't help but imagine all the time wasted turning the "wrong" knobs and trying the "wrong" things- i.e. tweaks to models that didn't pan out as hoped and that never get published. Having a "calculus of deep learning" would save a lot of the time wasted fumbling around in the dark only to come up with a dead end. Experiments could be guided by principle, people would know what is promising and what "might be worth a try". New researchers or outsiders would have a chance to get a grip on the gigantic bibliography armed with analytical tools, rather than forced to painstakingly develop an intuition that can be hardly communicated to their peers. The trouble is that there is nothing like that "deep learning calculus" yet and the whole field seems to me (as an outsider, of course) to be much more interested in finding what works, than understanding why it works.