3 ms·
It concerns me a bit that at some point, there are fundamentals concerning simple yet effective approaches that may be forgotten (or a least not taught) in favo
by thearn4 9y ago
It concerns me a bit that at some point, there are fundamentals concerning simple yet effective approaches that may be forgotten (or a least not taught) in favor of black box neural nets, like Kalman filtering, PID control, etc.
Institutional memory is something of a weak point in the software industry compared with other fields.
- markvdb 9y agoSimilar "institutional memory loss" happened in mechanics. On many occasions, it's not so much the theory as the fluency putting it in practice that might be lost...
- sjg007 9y agoI mean that's math... not as likely to get lost. Now designing a NN to perform kalmann filtering seems interesting ways maybe it would lead to insight in how the network operates... e.g more of a grey box
- clickok 9y agoPart of the solution, I think, is making implementations of existing techniques available (and of course, rewriting them in Rust). Deep learning is popular because it's so widely applicable that it justifies the effort of learning to use it: deep nets work with image processing and robot control, and it's kinda unusual to have the mostly the same bag of tricks such different tasks. But a reliable way to improve your performance (or at least write a publishable paper) is to examine why a specific technique worked, rewrite it as a differentiable function or finite automaton, and then implement it as a component in a deep net. So it's a cause for concern, but also an opportunity for academic arbitrage. I think it will work out eventually, but I become uncertain when I meet machine learning "experts" who are only familiar with deep learning and unwilling to consider anything else.