4 ms·
There is a subfield that does serious mathematics, but their results are usually far removed from the state of the art stuff. Their results usually look like "a
by nightcracker 7y ago
There is a subfield that does serious mathematics, but their results are usually far removed from the state of the art stuff. Their results usually look like "a neural network with 1 hidden layer is a universal approximator" or "exponential expected convergence speed on linear relations for <X>".
- SpaceManNabs 7y agoNot necessarily true. The polynomial time for the escape of saddle points by using stochastic noise is pretty practical.
- nightcracker 7y agoSorry, my statement was too strong. I edited it to "usually".
- p1esk 7y agoHow in the world is this "pretty practical"? It would be if people used this theorem to come up with an idea of SGD, but that's not what happened. SGD appeared as a way to overcome the practical constraint of computing full GD. Not to mention that "polynomial time" is meaningless to any practitioner.
- SpaceManNabs 7y agoit did come with a new online update for orthogonal tensor decomposition using higher order moments and with comments on NP-hardness for 4th order and higher. In addition, it came with tricks with how much noise to inject in certain situations. "How much noise do you need is enough to escape?" which is pretty practical