4 ms·
> Since when did we become convinced deep learning achieves "global optimality" as they put it. There are a couple of papers over the past few years arguing/pr
by gwern 9y ago
> Since when did we become convinced deep learning achieves "global optimality" as they put it.
There are a couple of papers over the past few years arguing/proving under various models that the local optima in deep models either are or are within epsilon of the global optima and this is why SGD reaching only local optima results in models that still work well. I assume these results are covered in OP (haven't read it yet, just answering your question).
- catnaroek 9y ago> There are a couple of papers over the past few years arguing/proving under various models that the local optima in deep models either are or are within epsilon of the global optima I don't care so much for the “arguing” part, but the “proving” part got me interested. ML methods reliably being able to get epsilon-close to global optima is a BFD. What mathematical technology was used to establish these results? Could you provide links to those papers?