5 ms·
Can somebody explain why this is interesting computationally-wise? It looks like standard derivation to me.
by chombier 9y ago
Can somebody explain why this is interesting computationally-wise? It looks like standard derivation to me.
- nimish 9y agoNormally reverse mode automatic differentiation is used and is called backpropagation since everything old has to have a new and shiny name in machine learning a/k/a continuous optimization ;)
- lr4444lr 9y agoI concur. When I first started taking online coursework in machine learning, I was in almost every unit asking myself, didn't I cover this already in my non-linear optimization class at Uni.?
- laichzeit0 9y agoThe main difference is that a lot of practitioners using ML have very little advanced mathematical background. Let's be honest, you can't understand much of optimization without at least multivariable calculus and linear algebra. I think this is a very bad thing, but basically what happens when things become "commoditized". I feel it's the same with web developers who have no formal background in Computer Science. All these frameworks reinventing "paradigms" that have been studied since the 60's.
- sidlls 9y agoI'm not sure what your point is. People use tools without understanding the underlying principles all the time, frequently in ways that are more (tangibly) practical and interesting than a person who does would use them.
- mhb 9y agoPresumably his point is that you can be more effective at something if you understand it at a level one level below the one at which you are working.
- sidlls 9y agoI'd say that depends entirely on context. It also has diminishing value each "level" one goes. I mean, understanding quantum degeneracy pressure doesn't make physicists more effective at being solid.
- chombier 9y agoI think his point is about reinventing the wheel due to insufficient theoretical background.
- fjdlwlv 9y agoML researchers have the math background.
- thearn4 9y agoI have had the same thoughts too. What also had me a bit puzzled in the ML community is the lack of discussion on local optimality, and constraint/Lagrangian loss functions. But there is probably a subtlety that I'm missing.
- thearn4 9y agoThe engineers (aerospace, etc) also prefer the term "adjoint mode"
- fjdlwlv 9y agoIt's the first open source implementation compatible with the modern ecosystem. Software development isn't about novel math, it's about novel applications and deployments