4 ms·

Deep Learning is currently an empirical science guided by intuition of practitioners. A main principle in experimental sciences is that a theory without predict

by nora4 8y ago

Deep Learning is currently an empirical science guided by intuition of practitioners. A main principle in experimental sciences is that a theory without predictive power is not considered a full-fledged theory. As such, unless they are interesting predictions coming from their theory (rather than only barely justifying existing empirically observed phenomena), this is just speculative theory that I would not use the phrase "Foundations Built" for. As an example of this general litmus test for a theory see e.g. Eddington's confirmation of GR: https://en.wikipedia.org/wiki/Tests_of_general_relativity#Deflection_of_light_by_the_Sun https://en.wikipedia.org/wiki/Tests_of_general_relativity#De... . If there are hitherto unknown phenomena in DL predicted by this theory then I'd stand corrected and concede that there may be something to these theories.

fizx 8y ago
Read this as "some foundations built". Even that is much-needed progress.
- hyperpallium 8y ago
  Yeah, it's not foundations like elementary or axiomatic. An actual theory of ANN and then of NNN will be of more momentous import than of any previous phenomena, will usher in a fundamentally transformative age where we understand ourselves, and will surely require a fundamental breakthrough in pure mathematics... probably a great many.
matt4077 8y ago
The article seems pretty upfront about the fact that we are nowhere near a solid theory. One of many quotes to that effect: “ […] beginning to build the rudiments of a theory of neural networks.” The main point seems to be why such a theory is highly desirable. To that end, it does a far better job than the somehat tired Feynman aphorism about good theories “giving us more than we give them”. Namely that the current state of the art of “intuition” and vast trial-and-error runs being somewhat embarrassing for a field so closely related to math and statistics.

by nora4 8y ago

fizx 8y ago

Read this as "some foundations built". Even that is much-needed progress.

hyperpallium 8y ago
Yeah, it's not foundations like elementary or axiomatic. An actual theory of ANN and then of NNN will be of more momentous import than of any previous phenomena, will usher in a fundamentally transformative age where we understand ourselves, and will surely require a fundamental breakthrough in pure mathematics... probably a great many.

matt4077 8y ago

The article seems pretty upfront about the fact that we are nowhere near a solid theory. One of many quotes to that effect: “ […] beginning to build the rudiments of a theory of neural networks.” The main point seems to be why such a theory is highly desirable. To that end, it does a far better job than the somehat tired Feynman aphorism about good theories “giving us more than we give them”. Namely that the current state of the art of “intuition” and vast trial-and-error runs being somewhat embarrassing for a field so closely related to math and statistics.