> Do you have any sense of whether tricks like momentum which afaik came out of improving SGD for neural network training have found application in other arenas

> Do you have any sense of whether tricks like momentum which afaik came out of improving SGD for neural network training have found application in other arenas where batching is less reasonable? I thought Yurii Nesterov came up with momentum (or at least proved certain classes of momentum strategies optimal)?