3 ms·
Both RProp and Conjugate Gradient are descent methods-- they find a path to a local minima from the starting configuration. They do not help with finding the gl
by mierle 14y ago
Both RProp and Conjugate Gradient are descent methods-- they find a path to a local minima from the starting configuration. They do not help with finding the global minimum.
Using simulated annealing to guide random initializations can help find a better minima, but to get the global minima with simulated annealing takes an inordinate amount of time.
- stiff 14y ago(Poor) local minima can be sometimes reached in back propagation due to a bad setting of the learning rate and RProp does help with that. You can also repeat the learning process using one of those algorithms many times with different initial random weights and use the results of the best attempt. I think this most often suffices in practical applications of NN for getting decent results, you are right I got some of this wrong, I basically wanted to say that backprop is no longer state of the art in training and that optimization is the relatively easy part of learning.