5 ms·
> We argue complexity conceals underlying regularity, and that deep learning will indeed admit a scientific theory That would be amazing, but personally I’m sk
by le-mark 6mo ago
> We argue complexity conceals underlying regularity, and that deep learning will indeed admit a scientific theory
That would be amazing, but personally I’m skeptical.
- slashdave 6mo agoThere is an analogy with statistical mechanics. It's not crazy.
- wwarner 6mo agoSane & interesting enough to have been disproven, by Boaz Barak iirc. Maybe not surprising since simulated annealing never achieved the results of gradient descent + backprop.
- slashdave 6mo agoYou might be trying to be too literal. What makes statistical mechanics so brilliant is that it takes first principle ideas (particle energies + ensemble) to derive macroscopic thermodynamic rules, all of which were originally derived from observation. What the OP is proposing is a mathematical analysis of SGD + generic deep learning architectures might be able to derive the rules we have empirically derived from experiments in model training.
- seydor 6mo agobiology thrives in complexity though; yet all the electrons are identical