8 ms·

I question this meme that deep learning has less mathematical elegance or interpretability than other machine learning model. It's simply not true. At this poin

I question this meme that deep learning has less mathematical elegance or interpretability than other machine learning model. It's simply not true. At this point, it reads like a mindless copypasta with the possible political intention to reduce AI funding and use, such as the EU regulation which requires all algorithms to "explain" their reasoning. Either that, or the author is just more comfortable with familiar old-school methods which makes them think that those methods are more elegant. This push towards elegance is unscientific and counterproductive, much like the mistaken push Einstein and co. made for deterministic interpretations of quantum mechanics. We can now understand that its motivation was simply due to a biased notion of elegance due to familiarity with classical mechanics. Examples of elegance in deep learning: The gating mechanisms of LSTMs (also adds to interpretability); structure of convolutions, i.e. weight sharing, translation invariance; the policy gradient theorem (Sutton 1999); dropout, its relationship to biology and variational inference; Generative Adversarial Networks, and connections to game theory, split-brain; the reparametrization trick; the log-derivative trick; connectionist temporal classification. These examples are only the surface. Depending on your specialization inside deep learning, you'd find many more. Examples of interpretability: Nguyen et al.'s Synthesizing preferred inputs to hidden neurons, Zeiler et al. convnet visualization, Guillaume Alain's linear probes of hidden layers, attention readouts in attentional models for machine translation or speech synthesis, and many more. Ultimately, deep learning methods are probabilistic, and decent deep learning engineers would be able to tell you why a model is doing what it's doing by printing probabilities and activation statistics, much like any other probabilistic machine learning model.

trevyn 9y ago
I'm starting to think that human resistance to change is a form of regularization.
shas3 9y ago
Elad's pretty clear in laying out how pre-deep learning denoising research is differently elegant compared to CNN, etc. You should also consider the context, SIAM- the applied math community, which has seen a lot of work in deep mathematical connections between PDEs, denoising, etc. The math is certainly more advanced than required for understanding deep learning and applying it to achieve better results.
- YeGoblynQueenne 9y ago
  Thanks for this comment, even if you're just stating the obvious. It's amazing the lack of humility some posters display on HN when it comes to discussing the work of professionals with experience in the field. (I'm not Elad or affiliated to him or his instutition btw).
  - backpropaganda 9y ago
    I may not have the accolades Elad has, but I'm not a random HN commenter. I speak from own experience in machine learning.
    - YeGoblynQueenne 9y ago
      It doesn't matter who you are and what you've done. Some respect to what others have achieved is always a good thing. Lots of people have "experience of machine learning" these days. And there's so much information out there anyway that you really don't need to know what you're talking about to leave a quick comment on a discussion board.
gaius 9y ago
requires all algorithms to "explain" their reasoning It's about the application. If you are detecting cat images on a social network, then a few false positives or negatives are totally no big deal. If your algo is making investment decisions or is being used as evidence in court then it is entirely reasonable to expect it to have a rock solid causal chain easily comprehensible by a layperson. The example in the article is noise reduction and the method was "we tweaked it until we got the results we wanted". In many fields that's called "overfitting" (or "fitting up") and is a big red flag.
- backpropaganda 9y ago
  All machine learning models are capable of overfitting. Deep learning does not have a special relationship to overfitting, and in fact, there is some evidence that the structural priors in complex deep networks actually inhibit overfitting and improve generalization [1]. Citing certain examples of misuse of deep learning by folks who "tweaked it till it worked" doesn't say anything about deep learning at large. The same team would do the same with a much less powerful model. Of course, being able to explain is a very valuable trait. But there's just no evidence that deep learning methods are any less amenable to interpretation than SVMs or decision trees. The latter models' "interpretation" is mostly stuff like "feature 543 and 632 were on" , while deep learning methods can not only do that, but can also synthesize examples of characteristics that the model looked for [2]. [1]: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 [2]: https://arxiv.org/abs/1605.09304 https://arxiv.org/abs/1605.09304
  - yorwba 9y ago
    Then requiring algorithms to explain their reasoning should give neural networks an edge over other methods, if they are really easier to interpret.
  - gaius 9y ago
    Deep learning does not have a special relationship to overfitting Yes and no. Let's say you have a time series and you know there is some pattern in it and you are interested in predicting what it will be at time t. A Fourier analysis would give you a formula to plug numbers into that anyone can understand. NN or DL might give you as good, or accounting for noise, even a slightly better prediction when backtested but now you need to explain "why did it do this" - to a financial regulator who is looking at insider trades, or a commission looking into why a piece of equipment failed killing the user (say, a self-driving car), or a newspaper who's noticed more people of race X are being turned down for mortgages. That's why I say it is dependent on the application of the algo.
throw_away_777 9y ago
While I have nothing against black-box models and believe that interpretability is over-rated, deep learning is less interpretable than other models. But compare something like a simple linear model with a deep model and the linear model is much easier to interpret, especially for someone without a mathematical background. You don't have to cite papers to give examples of how to interpret a linear model. A single decision tree is also much easier to understand and explain than a deep model. Where deep learning and tree ensemble methods excel is in accuracy and ease of use.
- backpropaganda 9y ago
  I take issue with precisely the notion that deep learning models are "black-box". They're pretty transparent, and just because people haven't gained adeptness in it yet says more about its cutting-edginess than about its interpretability. The average programmer at a tech company won't be able to tell us how a particular complex piece of code works, but that doesn't stop us from building complex software. Deep learning methods are also not off-the-shelf type algorithms. Using them does require knowledge of the domain. This doesn't fit with the "black-box" narrative. In fact, SVMs and DTs are black-boxes due to their off-the-shelf nature. (jk lol)
  - throw_away_777 9y ago
    The deep learning buzz-word applies to models which learn feature interactions in a fairly complicated manner. While it is possible to explain what a model is doing, the model is a black-box in the sense that without a computer you could not develop the model. The model is making a lot of decisions, and explaining why all those choices are made is not really feasible. In contrast, I could create a linear model by hand easily, it just won't be as good at complicated predictions. However I think the stigma against black-boxes is undeserved. The human brain is a black-box and no one argues against using it.
    - ansgri 9y ago
      The human brain is a black-box and no one argues against using it. Well, nobody's against using the brain, but the current trend in most subject domains is to avoid overrelying on intuition, and checking and supporting the conclusions using structured thinking methods, i.e. logic. That is, deep learning is an intuition, and you have to have a high-level explanatory/verification mechanism that would support or reject the answer.
maze-le 9y ago
>> This push towards elegance is unscientific I think you are right on that one. The Standard Model for example is absoluteley not elegant, it is a huge convoluted mess of parameters and constants (just google for "Standard Model Lagrangian"). Yet, it is the best we came up with, explaining the dynamics of fields and particles[0], that surround us. Correct answers don't have to be elegant per se, its nice when they are, but it's not a prerequisite. But, consider this: You can train a Neural Network to predict the distance of an object, thrown with velocity v_0, an angle of α, under the influcence of gravitational acceleration g. After a few hunderd rounds of training, a suited NN can reasonably predict the outcome of said experiment, with the input: v_0, α and g. And, as you pointed out, a researcher can explain to you why this is the case based on activation, feedback-loops, learning algorithm and other parameters. But neither the NN nor the researcher will be able to give you a rule like: "F=(G m_1 m_2)/(r^2)" to explain the underlying reasons for the object-trajectory dynamics. A Neural Network can give you answers and predictions, yes, but you are not able to incorporate them into a wider theory, since the output is always numerical in nature. It is also always tied to one specific fact you are interested in and cannot give you a generalization. [0]: In the model of QFT particles are also fields of a different kind.
- manquer 9y ago
  A Neural network will give you a relation which is approximation of that relation not just a numeric output . Also you should remember that while it looks elegant the Newtonian model is also an approximation and fails noticeably in cases where relativistic effects are significant
- candiodari 9y ago
  That may have something to do with the fact that even the biggest neural networks have a "cortex size" one or two orders of magnitude smaller than a cat and "exist" (number of impulses processed * amount of time that would take in an animal) for a few weeks. That means one should expect them to perform tasks about as well as a mouse brain can do them, between a third and halfway to adulthood. And yes, that level of cortical processing does not seem to support coming up with symbolic systems and attempting to use them to describe the world around them. Absolutely. Mice don't do that either.
js8 9y ago
Can you recommend some good books/papers that explain how exactly the hypothesis space of DNNs looks like? By no free lunch theorem, there has to be one.
flippyhead 9y ago
I've been thinking lately that the reason facilities of our minds are simply a reflection of physical reality but only one version of it most useful for surviving at the scales in which we exist and give the "reality problems" we've evolved to solve. The idea that our perceptions indicate some truth about reality becomes easier to discount. We may not quite be there but in terms of the reasoning facilities of our minds, we seem to have understood nearly all that can be understood about the universe. All the quantum stuff is simply not understandable using all that machinery but seems to require we run a virtual-machine like thing grafting reason onto weird probabilistic models. Given all this, I don't immediately assume the most "elegant" solution is really the best anymore for most of the big problems we're now trying to solve. At least in the domain of science.
eksemplar 9y ago
I think it comes down to politics, which is ultimately about convincing a public. The same public which has caused measles pandemics to rise again because enough of it believed a YouTube video. One example is in the child services. We've got enough digital historical data from child services, that we're able to build models, that let's us predict whether or not a child will have to be removed from abusive parents 5-10 years from now. This information is relatively useless though, because there is no way to explain how it works in a sense that the general public will accept. The public is probably going to adjust eventually, but until that happens, you'll see an increasing push for documentation on how a result was reached. Ironically an algorithm that explains how it ended up with the results that it did, isn't any more moral than an algorithm that didn't. I mean, you're losing you autonomy and freedom when the government knows what you'll do before you do either way.
- xamuel 9y ago
  >An algorithm that explains ... isn't any more moral Not sure what "moral" means, but two things: 1. If black boxes make unexplainable decisions, that gives too much power to the person who reads the box's answer, 2. If the child services algorithm explained its prediction involved e.g. alcoholism, then the parents could try to quit.
  - xherberta 9y ago
    Unfortunately, you can't quit alcoholism. Also troubling: what if the child service algo happens to predict that brown families are more abusive?
zamalek 9y ago
If the internet were around when microwave ovens were invented, there would have been much the same outcry. Deep learning is excessively simple and elegant, I can explain it in person in under 5 minutes to anyone who has done high school algebra.
- reader5000 9y ago
  Great. Explain why training converges in subexponential time despite the nonconvex cost function and does not overfit the data.
  - zamalek 9y ago
    You only need to know that if you are studying the field. A good explanation of deep learning is no more surprising to the general populace than general relativity is (spacetime isn't actually the surface of a balloon).
    - reader5000 9y ago
      That's not correct though, theres plenty of reasons why on an intuitive level neural networks should not work. Also, genetic algorithms are intuitively appealing but do not work well.
      - singham 9y ago
        Genetic Algorithms are used in designing of antenna arrays.
  - sprobertson 9y ago
    As you get closer to the right answer, the changes you need to make are smaller and harder to discover.
  - jph00 9y ago
    Because in high dimensional space there are partial saddle points, not local minima. Intuitively obvious, and shown theoretically in the last 12 months.
reader5000 9y ago
The article meant elegant in the sense of "nn training provably converges in subexponential time and doesnt overfit the data". E.g., like the theory for convex SVMs. Mathematically, nn training is an unproven algorithm, it just works empirically surprisingly well. It's analogous to the simplex method which worked well for years without theoretical justification.
Cybiote 9y ago
I disagree that his points are due to mere familiarity with "old-school methods". I view elegance as something that is simple in hindsight but only if with an appropriate background. These are simple movements that belie the expertise it took to get there. Older methods were proof heavy and required more mathematical sophistication but once you understood them, you could easily implement the idea. A lot of deep learning is comparatively mathematically simpler but there is often much incidental detail and folk knowledge that ends up being important but not mentioned. It is in this sense that one can call DL inelegant. A lot of that is because so many are racing to publish results, there's not much time to contrast with prior art or do much more than justify with often very expensive experiments with hastily scrawled descriptions. Lots of ideas generated without much context means many of them are quickly dropped and forgotten, sometimes without sufficient justification. > Examples of elegance in deep learning GANs, VAEs, gating, convolutions and weight sharing are indeed great ideas. IIRC Hinton's early papers did inspire Friston's influential work in theoretical neuroscience. However, translation invariance is double counting the advantages of convolutions. Policy gradients, the log-derivative and reparametrization tricks are independent of deep learning. Variational inference is more due to how far reaching that idea is if you want efficient generative models. Game theory...well lots of ideas are connected to it, including evolution and the dead simple weighted majority algorithm. Split brain is really reaching, especially since it's now looking to be one of those ideas that will need a decent amount of revision. That said, I agree that too much emphasis is placed on interpretability. If a function is too complex, then a human will simply be unable to fit it in their working memory. Nonetheless, the ability to introspect on some of a neural net's decision is vital. As you say, visualizations and mappings which project to a simpler function space but preserve most of the detail are just as possible with neural nets as with any other method. In this post I might come across as disparaging of deep learning, but this is certainly not my intention. There are lots of excellent papers which elegantly tackle difficult questions. They ask: what invariances and structures do we seek? How do we learn good representations we can sample from? How do we keep learning stable and achieve good gradient flow? How can we capture longer range correlations? These also offer insights that are not limited to deep learning. You just don't hear as much about them because they are not as shiny. And if you're looking for mathematical elegance that cuts to the heart of the matter, specially with the rising importance of generative models, you could hardly do worse than start from anything written by Shun-Ichi Amari. People always want to jump right to the shiny stuff and skip the basics, this is understandable but suboptimal in the long term, in any field. But a good compromise on this matter is the excellent free deep learning book by Goodfellow, Bengio and Courville.
Xcelerate 9y ago
> mistaken push Einstein and co. made for deterministic interpretations of quantum mechanics But really. Secretly we all expect one to exist below it all.
- singham 9y ago
  I have heard about Superdeterminism of Gerard t'Hooft. [1] https://physics.stackexchange.com/questions/34217/why-do-people-categorically-dismiss-some-simple-quantum-models https://physics.stackexchange.com/questions/34217/why-do-peo... [2] https://blogs.scientificamerican.com/critical-opalescence/does-some-deeper-level-of-physics-underlie-quantum-mechanics-an-interview-with-nobelist-gerard-e28099t-hooft/ https://blogs.scientificamerican.com/critical-opalescence/do...
YeGoblynQueenne 9y ago
>> Examples of elegance in deep learning: The gating mechanisms of LSTMs (also adds to interpretability); LSTMS amaze and dumbfound me in equal measures and I admire anyone who can understand them, at all. Could you please indulge my cluelessness and explain to me what is mathematically elegant about the gating mechanism in LSTMs and how it adds to interpretability?
- andrew3726 9y ago
  You might like this article[0]. It definitely helped me develop some intuition for LSTMs. 0: http://r2rt.com/written-memories-understanding-deriving-and-extending-the-lstm.html http://r2rt.com/written-memories-understanding-deriving-and-...
  - YeGoblynQueenne 9y ago
    Thanks, but I wanted to hear why the OP thinks LSTM gates are elegant and add to interpretability. The OP made a big todo about how the article above is misguided and the person who wrote it has a poor understanding of deep learning, so I'm assuming he or she has a very good understanding of the subject. I mean, I'd hate to believe the OP is just playing deep learning bingo for HN karma, you know?
kefka 9y ago
>I question this meme that deep learning has less mathematical elegance or interpretability than other machine learning model. It's simply not true. I know, right? Like, take this model I trained this morning. Here's the parameters it learned:[0.230948, 0.00000000014134, 0.1039402934, 0.000023001323, 0.00000000000005] I mean, what's "black-box" about that, really? You can instantly: (a) See exactly what the model is a representation of. (b) Figure out what data was used to train it. (c) Understand the connection between the training data and the learned model. It's not like the model has reduced a bunch of unfathomably complex numbers to another, equally unfathomable. You can tell exactly what it's doing- and, with some visualisation, it gets even better.Because then it's a curve. Everyone groks curves, right?Right, you guys? /s obviously.
- p1esk 9y ago
  I don't know what model you trained but if it only needs 5 parameters it's quite possible that you can figure out the corresponding features, and remove your sarcasm tags.
  - YeGoblynQueenne 9y ago
    It's perfectly possible to train a deep neural model with millions of features that outputs a vector of a handful of values, or even just a scalar. That's actually one of their strengths. [Full disclosure: that comment was originally mine from a different thread; I'm not affiliated with the OP in any way.]
    - kefka 9y ago
      Yes indeed. Your comment was sarcastically perfectly succinct and to the point. And it left a mark on me as being the best way to get across this "magic math" stuff encroaching into computer science. It does appear to be a very powerful tool, but with absolutely no introspection or any way to validate or invalidate content. Best one can hope, is that the dataset has no inherent biases ~ but even that's no claim the algo won't create a layer to make the biases, and then directly chose on biases. I probably should have cited it (was on mobile, but oh well :/ ). Well, I'll do it now: https://news.ycombinator.com/item?id=14219450 https://news.ycombinator.com/item?id=14219450 If anything, please consider this copy/edit as a sign of respect, and not of mockery :)
deepnotderp 9y ago
You forgot attention mechanisms, that's a huge one

trevyn 9y ago

I'm starting to think that human resistance to change is a form of regularization.

shas3 9y ago

Elad's pretty clear in laying out how pre-deep learning denoising research is differently elegant compared to CNN, etc. You should also consider the context, SIAM- the applied math community, which has seen a lot of work in deep mathematical connections between PDEs, denoising, etc. The math is certainly more advanced than required for understanding deep learning and applying it to achieve better results.

YeGoblynQueenne 9y ago
Thanks for this comment, even if you're just stating the obvious. It's amazing the lack of humility some posters display on HN when it comes to discussing the work of professionals with experience in the field. (I'm not Elad or affiliated to him or his instutition btw).
- backpropaganda 9y ago
  I may not have the accolades Elad has, but I'm not a random HN commenter. I speak from own experience in machine learning.
  - YeGoblynQueenne 9y ago
    It doesn't matter who you are and what you've done. Some respect to what others have achieved is always a good thing. Lots of people have "experience of machine learning" these days. And there's so much information out there anyway that you really don't need to know what you're talking about to leave a quick comment on a discussion board.

gaius 9y ago

requires all algorithms to "explain" their reasoning It's about the application. If you are detecting cat images on a social network, then a few false positives or negatives are totally no big deal. If your algo is making investment decisions or is being used as evidence in court then it is entirely reasonable to expect it to have a rock solid causal chain easily comprehensible by a layperson. The example in the article is noise reduction and the method was "we tweaked it until we got the results we wanted". In many fields that's called "overfitting" (or "fitting up") and is a big red flag.

backpropaganda 9y ago
All machine learning models are capable of overfitting. Deep learning does not have a special relationship to overfitting, and in fact, there is some evidence that the structural priors in complex deep networks actually inhibit overfitting and improve generalization [1]. Citing certain examples of misuse of deep learning by folks who "tweaked it till it worked" doesn't say anything about deep learning at large. The same team would do the same with a much less powerful model. Of course, being able to explain is a very valuable trait. But there's just no evidence that deep learning methods are any less amenable to interpretation than SVMs or decision trees. The latter models' "interpretation" is mostly stuff like "feature 543 and 632 were on" , while deep learning methods can not only do that, but can also synthesize examples of characteristics that the model looked for [2]. [1]: https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 [2]: https://arxiv.org/abs/1605.09304 https://arxiv.org/abs/1605.09304
- yorwba 9y ago
  Then requiring algorithms to explain their reasoning should give neural networks an edge over other methods, if they are really easier to interpret.
- gaius 9y ago
  Deep learning does not have a special relationship to overfitting Yes and no. Let's say you have a time series and you know there is some pattern in it and you are interested in predicting what it will be at time t. A Fourier analysis would give you a formula to plug numbers into that anyone can understand. NN or DL might give you as good, or accounting for noise, even a slightly better prediction when backtested but now you need to explain "why did it do this" - to a financial regulator who is looking at insider trades, or a commission looking into why a piece of equipment failed killing the user (say, a self-driving car), or a newspaper who's noticed more people of race X are being turned down for mortgages. That's why I say it is dependent on the application of the algo.

throw_away_777 9y ago

While I have nothing against black-box models and believe that interpretability is over-rated, deep learning is less interpretable than other models. But compare something like a simple linear model with a deep model and the linear model is much easier to interpret, especially for someone without a mathematical background. You don't have to cite papers to give examples of how to interpret a linear model. A single decision tree is also much easier to understand and explain than a deep model. Where deep learning and tree ensemble methods excel is in accuracy and ease of use.

backpropaganda 9y ago
I take issue with precisely the notion that deep learning models are "black-box". They're pretty transparent, and just because people haven't gained adeptness in it yet says more about its cutting-edginess than about its interpretability. The average programmer at a tech company won't be able to tell us how a particular complex piece of code works, but that doesn't stop us from building complex software. Deep learning methods are also not off-the-shelf type algorithms. Using them does require knowledge of the domain. This doesn't fit with the "black-box" narrative. In fact, SVMs and DTs are black-boxes due to their off-the-shelf nature. (jk lol)
- throw_away_777 9y ago
  The deep learning buzz-word applies to models which learn feature interactions in a fairly complicated manner. While it is possible to explain what a model is doing, the model is a black-box in the sense that without a computer you could not develop the model. The model is making a lot of decisions, and explaining why all those choices are made is not really feasible. In contrast, I could create a linear model by hand easily, it just won't be as good at complicated predictions. However I think the stigma against black-boxes is undeserved. The human brain is a black-box and no one argues against using it.
  - ansgri 9y ago
    The human brain is a black-box and no one argues against using it. Well, nobody's against using the brain, but the current trend in most subject domains is to avoid overrelying on intuition, and checking and supporting the conclusions using structured thinking methods, i.e. logic. That is, deep learning is an intuition, and you have to have a high-level explanatory/verification mechanism that would support or reject the answer.

maze-le 9y ago

>> This push towards elegance is unscientific I think you are right on that one. The Standard Model for example is absoluteley not elegant, it is a huge convoluted mess of parameters and constants (just google for "Standard Model Lagrangian"). Yet, it is the best we came up with, explaining the dynamics of fields and particles[0], that surround us. Correct answers don't have to be elegant per se, its nice when they are, but it's not a prerequisite. But, consider this: You can train a Neural Network to predict the distance of an object, thrown with velocity v_0, an angle of α, under the influcence of gravitational acceleration g. After a few hunderd rounds of training, a suited NN can reasonably predict the outcome of said experiment, with the input: v_0, α and g. And, as you pointed out, a researcher can explain to you why this is the case based on activation, feedback-loops, learning algorithm and other parameters. But neither the NN nor the researcher will be able to give you a rule like: "F=(G m_1 m_2)/(r^2)" to explain the underlying reasons for the object-trajectory dynamics. A Neural Network can give you answers and predictions, yes, but you are not able to incorporate them into a wider theory, since the output is always numerical in nature. It is also always tied to one specific fact you are interested in and cannot give you a generalization. [0]: In the model of QFT particles are also fields of a different kind.

manquer 9y ago
A Neural network will give you a relation which is approximation of that relation not just a numeric output . Also you should remember that while it looks elegant the Newtonian model is also an approximation and fails noticeably in cases where relativistic effects are significant
candiodari 9y ago
That may have something to do with the fact that even the biggest neural networks have a "cortex size" one or two orders of magnitude smaller than a cat and "exist" (number of impulses processed * amount of time that would take in an animal) for a few weeks. That means one should expect them to perform tasks about as well as a mouse brain can do them, between a third and halfway to adulthood. And yes, that level of cortical processing does not seem to support coming up with symbolic systems and attempting to use them to describe the world around them. Absolutely. Mice don't do that either.

js8 9y ago

Can you recommend some good books/papers that explain how exactly the hypothesis space of DNNs looks like? By no free lunch theorem, there has to be one.

flippyhead 9y ago

I've been thinking lately that the reason facilities of our minds are simply a reflection of physical reality but only one version of it most useful for surviving at the scales in which we exist and give the "reality problems" we've evolved to solve. The idea that our perceptions indicate some truth about reality becomes easier to discount. We may not quite be there but in terms of the reasoning facilities of our minds, we seem to have understood nearly all that can be understood about the universe. All the quantum stuff is simply not understandable using all that machinery but seems to require we run a virtual-machine like thing grafting reason onto weird probabilistic models. Given all this, I don't immediately assume the most "elegant" solution is really the best anymore for most of the big problems we're now trying to solve. At least in the domain of science.

eksemplar 9y ago

I think it comes down to politics, which is ultimately about convincing a public. The same public which has caused measles pandemics to rise again because enough of it believed a YouTube video. One example is in the child services. We've got enough digital historical data from child services, that we're able to build models, that let's us predict whether or not a child will have to be removed from abusive parents 5-10 years from now. This information is relatively useless though, because there is no way to explain how it works in a sense that the general public will accept. The public is probably going to adjust eventually, but until that happens, you'll see an increasing push for documentation on how a result was reached. Ironically an algorithm that explains how it ended up with the results that it did, isn't any more moral than an algorithm that didn't. I mean, you're losing you autonomy and freedom when the government knows what you'll do before you do either way.

xamuel 9y ago
>An algorithm that explains ... isn't any more moral Not sure what "moral" means, but two things: 1. If black boxes make unexplainable decisions, that gives too much power to the person who reads the box's answer, 2. If the child services algorithm explained its prediction involved e.g. alcoholism, then the parents could try to quit.
- xherberta 9y ago
  Unfortunately, you can't quit alcoholism. Also troubling: what if the child service algo happens to predict that brown families are more abusive?

zamalek 9y ago

If the internet were around when microwave ovens were invented, there would have been much the same outcry. Deep learning is excessively simple and elegant, I can explain it in person in under 5 minutes to anyone who has done high school algebra.

reader5000 9y ago
Great. Explain why training converges in subexponential time despite the nonconvex cost function and does not overfit the data.
- zamalek 9y ago
  You only need to know that if you are studying the field. A good explanation of deep learning is no more surprising to the general populace than general relativity is (spacetime isn't actually the surface of a balloon).
  - reader5000 9y ago
    That's not correct though, theres plenty of reasons why on an intuitive level neural networks should not work. Also, genetic algorithms are intuitively appealing but do not work well.
    - singham 9y ago
      Genetic Algorithms are used in designing of antenna arrays.
- sprobertson 9y ago
  As you get closer to the right answer, the changes you need to make are smaller and harder to discover.
- jph00 9y ago
  Because in high dimensional space there are partial saddle points, not local minima. Intuitively obvious, and shown theoretically in the last 12 months.

reader5000 9y ago

The article meant elegant in the sense of "nn training provably converges in subexponential time and doesnt overfit the data". E.g., like the theory for convex SVMs. Mathematically, nn training is an unproven algorithm, it just works empirically surprisingly well. It's analogous to the simplex method which worked well for years without theoretical justification.

Cybiote 9y ago

I disagree that his points are due to mere familiarity with "old-school methods". I view elegance as something that is simple in hindsight but only if with an appropriate background. These are simple movements that belie the expertise it took to get there. Older methods were proof heavy and required more mathematical sophistication but once you understood them, you could easily implement the idea. A lot of deep learning is comparatively mathematically simpler but there is often much incidental detail and folk knowledge that ends up being important but not mentioned. It is in this sense that one can call DL inelegant. A lot of that is because so many are racing to publish results, there's not much time to contrast with prior art or do much more than justify with often very expensive experiments with hastily scrawled descriptions. Lots of ideas generated without much context means many of them are quickly dropped and forgotten, sometimes without sufficient justification. > Examples of elegance in deep learning GANs, VAEs, gating, convolutions and weight sharing are indeed great ideas. IIRC Hinton's early papers did inspire Friston's influential work in theoretical neuroscience. However, translation invariance is double counting the advantages of convolutions. Policy gradients, the log-derivative and reparametrization tricks are independent of deep learning. Variational inference is more due to how far reaching that idea is if you want efficient generative models. Game theory...well lots of ideas are connected to it, including evolution and the dead simple weighted majority algorithm. Split brain is really reaching, especially since it's now looking to be one of those ideas that will need a decent amount of revision. That said, I agree that too much emphasis is placed on interpretability. If a function is too complex, then a human will simply be unable to fit it in their working memory. Nonetheless, the ability to introspect on some of a neural net's decision is vital. As you say, visualizations and mappings which project to a simpler function space but preserve most of the detail are just as possible with neural nets as with any other method. In this post I might come across as disparaging of deep learning, but this is certainly not my intention. There are lots of excellent papers which elegantly tackle difficult questions. They ask: what invariances and structures do we seek? How do we learn good representations we can sample from? How do we keep learning stable and achieve good gradient flow? How can we capture longer range correlations? These also offer insights that are not limited to deep learning. You just don't hear as much about them because they are not as shiny. And if you're looking for mathematical elegance that cuts to the heart of the matter, specially with the rising importance of generative models, you could hardly do worse than start from anything written by Shun-Ichi Amari. People always want to jump right to the shiny stuff and skip the basics, this is understandable but suboptimal in the long term, in any field. But a good compromise on this matter is the excellent free deep learning book by Goodfellow, Bengio and Courville.

Xcelerate 9y ago

> mistaken push Einstein and co. made for deterministic interpretations of quantum mechanics But really. Secretly we all expect one to exist below it all.

singham 9y ago
I have heard about Superdeterminism of Gerard t'Hooft. [1] https://physics.stackexchange.com/questions/34217/why-do-people-categorically-dismiss-some-simple-quantum-models https://physics.stackexchange.com/questions/34217/why-do-peo... [2] https://blogs.scientificamerican.com/critical-opalescence/does-some-deeper-level-of-physics-underlie-quantum-mechanics-an-interview-with-nobelist-gerard-e28099t-hooft/ https://blogs.scientificamerican.com/critical-opalescence/do...

YeGoblynQueenne 9y ago

>> Examples of elegance in deep learning: The gating mechanisms of LSTMs (also adds to interpretability); LSTMS amaze and dumbfound me in equal measures and I admire anyone who can understand them, at all. Could you please indulge my cluelessness and explain to me what is mathematically elegant about the gating mechanism in LSTMs and how it adds to interpretability?

andrew3726 9y ago
You might like this article[0]. It definitely helped me develop some intuition for LSTMs. 0: http://r2rt.com/written-memories-understanding-deriving-and-extending-the-lstm.html http://r2rt.com/written-memories-understanding-deriving-and-...
- YeGoblynQueenne 9y ago
  Thanks, but I wanted to hear why the OP thinks LSTM gates are elegant and add to interpretability. The OP made a big todo about how the article above is misguided and the person who wrote it has a poor understanding of deep learning, so I'm assuming he or she has a very good understanding of the subject. I mean, I'd hate to believe the OP is just playing deep learning bingo for HN karma, you know?

kefka 9y ago

>I question this meme that deep learning has less mathematical elegance or interpretability than other machine learning model. It's simply not true. I know, right? Like, take this model I trained this morning. Here's the parameters it learned:[0.230948, 0.00000000014134, 0.1039402934, 0.000023001323, 0.00000000000005] I mean, what's "black-box" about that, really? You can instantly: (a) See exactly what the model is a representation of. (b) Figure out what data was used to train it. (c) Understand the connection between the training data and the learned model. It's not like the model has reduced a bunch of unfathomably complex numbers to another, equally unfathomable. You can tell exactly what it's doing- and, with some visualisation, it gets even better.Because then it's a curve. Everyone groks curves, right?Right, you guys? /s obviously.

p1esk 9y ago
I don't know what model you trained but if it only needs 5 parameters it's quite possible that you can figure out the corresponding features, and remove your sarcasm tags.
- YeGoblynQueenne 9y ago
  It's perfectly possible to train a deep neural model with millions of features that outputs a vector of a handful of values, or even just a scalar. That's actually one of their strengths. [Full disclosure: that comment was originally mine from a different thread; I'm not affiliated with the OP in any way.]
  - kefka 9y ago
    Yes indeed. Your comment was sarcastically perfectly succinct and to the point. And it left a mark on me as being the best way to get across this "magic math" stuff encroaching into computer science. It does appear to be a very powerful tool, but with absolutely no introspection or any way to validate or invalidate content. Best one can hope, is that the dataset has no inherent biases ~ but even that's no claim the algo won't create a layer to make the biases, and then directly chose on biases. I probably should have cited it (was on mobile, but oh well :/ ). Well, I'll do it now: https://news.ycombinator.com/item?id=14219450 https://news.ycombinator.com/item?id=14219450 If anything, please consider this copy/edit as a sign of respect, and not of mockery :)

deepnotderp 9y ago

You forgot attention mechanisms, that's a huge one