4 ms·

First comment I make on HN, but I feel compelled to say something. This post reeks of the author piping on about the "good old days" of mathematical elegance.

First comment I make on HN, but I feel compelled to say something. This post reeks of the author piping on about the "good old days" of mathematical elegance. Firstly, mathematical elegance is not an objective metric by any means. Many mathematicians disagree on what results are most elegant, though there are many commonalities as well. Secondly, all of machine learning is quite elegant. I'm by no means an expert since I only took two courses in my fourth year that were surveys of many methods (quite mathematically rigorous btw) but I've taken enough math to be able to judge what is elegant or not. As the author himself says, he has slightly modified his research methods. These days you'd be a fool to ignore deep learning, whether you have a deep understanding of it or not.

matt4077 9y ago
I felt the article was quite fair in acknowledging the intriguing results of deep learning, and decidedly arguing against ignoring them. "Elegance" can indeed not be defined, but I believe the misunderstanding here is more about the point of view: Deep neural networks are plenty elegant when looked at from an ML POV. But, if you're an expert in, for example, image processing, using a DNN as a tool, the solution it may give you won't be "elegant" by anybody's definition. It is, after all, a repetitive formula with X million arbitrary floats. Previous ML methods usually resulted in models, or formulas, that were small enough to "grasp" intuitively. The "beauty", or "elegance" was that often, you could find connections from terms in your model to the real world.
- ganfortran 9y ago
  > Previous ML methods usually resulted in models, or formulas, that were small enough to "grasp" intuitively. Which is not true in practice. SVM in reality is an instance based method, with thousands to 10 thousands high-dimensional vectors as its 'parameters'. Random forests, which is famous for its ease of interpretability is no easy meal either when you end up with 100s of trees with outrageous branches. Not mention that real world model are embarrassingly complex ensemble of smaller but still complex models.

matt4077 9y ago

I felt the article was quite fair in acknowledging the intriguing results of deep learning, and decidedly arguing against ignoring them. "Elegance" can indeed not be defined, but I believe the misunderstanding here is more about the point of view: Deep neural networks are plenty elegant when looked at from an ML POV. But, if you're an expert in, for example, image processing, using a DNN as a tool, the solution it may give you won't be "elegant" by anybody's definition. It is, after all, a repetitive formula with X million arbitrary floats. Previous ML methods usually resulted in models, or formulas, that were small enough to "grasp" intuitively. The "beauty", or "elegance" was that often, you could find connections from terms in your model to the real world.

ganfortran 9y ago
> Previous ML methods usually resulted in models, or formulas, that were small enough to "grasp" intuitively. Which is not true in practice. SVM in reality is an instance based method, with thousands to 10 thousands high-dimensional vectors as its 'parameters'. Random forests, which is famous for its ease of interpretability is no easy meal either when you end up with 100s of trees with outrageous branches. Not mention that real world model are embarrassingly complex ensemble of smaller but still complex models.