3 ms·
I feel like there’s something more interesting going on here than the author is giving credit to. Most tools you can start by solving simple problems with, and
by wokwokwok 6y ago
I feel like there’s something more interesting going on here than the author is giving credit to.
Most tools you can start by solving simple problems with, and gradually work up the complexity of the problem you’re addressing until it does “something useful”.
This is a good way to learn what you can, can’t and should use a tool for.
Deep learning is problematic though; solving trivial tasks is actually quite difficult, and often the “something useful” level of sophistication means copy pasting someone else’s paper and tweaking it a bit and kind of vaguely hoping something you do makes any difference.
Being able to solve trivial problems with neural networks is really important, and useful; not because it’s a good solution, but because it means you can see what happens when you try it out.
The problem with this post isn’t the conclusion; the author is quite right. You can solve this problem in many ways, maybe NN aren’t the best for this kind of trivial problem.
...but, if you want to solve harder problems with more than random trial and error tweaking parameters, solving easy problems with the same tool is a good way to learn how.
...and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively.
- roenxi 6y agoI can only agree. It is also notable that pretty much the entire repertoire of debugging techniques from programming land are useless. Attempting to implementing a paper from scratch is an interesting experience. (1) implement what they describe (2) it doesn't work (3) ... well, that was fun. Project over. Very different experience from trying to implement quicksort, even though the extensions to a basic neural net often aren't conceptually much more complicated.
- disgruntledphd2 6y ago> and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively. What do you mean by this? In what context? Like, the twenty year old models are still running in credit risk and insurance, so I'm confused if you mean in ML/statistical modelling.
- wokwokwok 6y agoYou know, driving cars, like the article was talking about? ...or, NLP, audio & image recognition, recommendations... come on. It’s not controversial.
- mattkrause 6y agoI do sorta wonder about that. The deep learning "revolution" corresponded with an exponential growth in the time/effort/money being thrown at these problems and the amount of data available to do so. In an alternate universe, could everyone be going crazy about kernel machines?
- disgruntledphd2 6y agoMaybe. GPU's are definitely a big part of why this stuff has improved, as you can train much, much faster on larger datasets which is going to improve performance. NN's are super flexible though, and I'm not sure you'd have gotten the same level of performance out of other methods. Interesting question though.
- disgruntledphd2 6y ago> recommendations A lot of this is standard statistical methods, much of which are much older than twenty years. Really, that's the part that threw me, twenty years ago everyone was going crazy for SVM's, which is still machine learning, but the features were definitely hand crafted. I think deep learning has been super successful with unstructured data, but for tabular data it's pretty much a wash between NN's and boosted trees or generalised additive models. You do need some flexibility in your function approximation, but not as much as people commonly believe.