3 ms·
That isn't what I'm arguing at all.
by moultano 6y ago
That isn't what I'm arguing at all.
- candiodari 6y agoI think this is a way to really piss off statisticians: Stop talking about minima. Stop talking about how your optimization algorithm behaves around a minimum. Nobody ever trains their model remotely close to convergence. ADAM doesn’t even provably converge. All real models diverge! You are nowhere close to a minimum! Stop talking about minima already goddamnit! Why even think about minima?! Minima are a myth! Everybody proves their results for minima of a convex function. Actually arguing for divergent models. I mean, I love it, just because it's so very very cruel.