I don't know what OP means by saying SGD. If he/she means it is Gradient-Based optimization algorithm as a whole, it is definitely a challenging and open questi

I don't know what OP means by saying SGD. If he/she means it is Gradient-Based optimization algorithm as a whole, it is definitely a challenging and open question, and a lot of researchers trying to improve them.