3 ms·
Stupid Q: Assuming "gradient descent" is roughly similar to the classical "steepest descent" optimization algorithm (???), why aren't deep learning researchers
by jabl 9y ago
Stupid Q: Assuming "gradient descent" is roughly similar to the classical "steepest descent" optimization algorithm (???), why aren't deep learning researchers looking into other more advanced algorithms from classical non-linear optimization theory. Like, say, (preconditioned) conjugate gradient, or quasi-Newton methods such as BFGS?