I always wonder how one would guarantee stability / convergence when learning in the field. Sounds great, but how can this be pulled off in a reliable way? Just

I always wonder how one would guarantee stability / convergence when learning in the field. Sounds great, but how can this be pulled off in a reliable way? Just dragging down the learning rate does not suffice in my oppinion, as for effective training you should be at the boundary of stability and greedy updates from my experience