3 ms·
Specifically for neural networks, is there any alternative for backpropagation and gradient descent which guarantee finding the global minimum?
by yu3zhou4 3y ago
Specifically for neural networks, is there any alternative for backpropagation and gradient descent which guarantee finding the global minimum?
- b3kart 3y agoUnlikely given the dimensionality and complexity of the search space. Besides, we probably don’t even care about the global minimum: the loss we’re optimising is a proxy for what we really care about (performance on unseen data). Counter-example: a model that perfectly memorises the training data can be globally optimal (ignoring regularization), but is not very useful.