2 ms·
>>> "None of these shortcomings is likely to be solved soon." >> There's no evidence for this, and I think it underestimates the large and well-funded machine
by YeGoblynQueenne 9y ago
>>> "None of these shortcomings is likely to be solved soon."
>> There's no evidence for this, and I think it underestimates the large and well-funded machine learning community.
Well- deep learning has been around for a few decades now. LeCun's LeNet-5 is already 20 years old. LSTM is 21 (from the Hochreiter and Schmidhuber paper). Those are the two best-known today architectures, but work on deep nets goes way back to the '80s.
So those are old algorithms and techniques and they've had the same limitations for 20+ years. What makes them "likely to be solved soon" now?
- skybrian 9y agoA lot more attention. The field seems to be moving pretty fast and doesn't seem to be stuck. But I wouldn't say "likely" or "unlikely" but rather that it's uncertain.
- YeGoblynQueenne 9y agoTwo problems with that. First, what determines whether a problem is close to solving is usually not the amount of interest in it but, rather, its hardness. See for instance P = NP, or the drive to find a cure for "cancer". Neither has been solved yet despite considerable effort and investment (particularly in cancer cures). The limitations of deep learning are a similarly hard problem- they're "baked in" to its DNA so it's a very hard problem how to overcome them [1]. Second, yep- the field is stuck, well and truly stuck. The fact that LSTM and CNN are 20+ years old is telling. The vast majority of ANN research today is basically trying small tweaks to old ideas to see if something will come out of them. There is very little incentive to research something radically different- because people can already make a very lucrative career with little timid architectural tweaks and researching something new and untried is risky. So even if just throwing a lot of money and minds to a hard problem could just make it go away, nobody is really trying. It will be a long time before a majority of researchers recognise the limitations of deep learning and in general statistical machine learning and do something about them. ___________________ [1] The article links to an article by François Chollet of Keras. That's the second part of two. The first one is orders of magnitude better informed than the article above and recommended background reading in any discussion about the limitations of deep learning: https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html