>when they say "given the simplicity of the method" I think they meant the implementation, in the end, is not complicated: there are a few variables that get u

>when they say "given the simplicity of the method" I think they meant the implementation, in the end, is not complicated: there are a few variables that get updated according to some formulas, and that's it; there's no branching, no need for fancy data structures, etc. The method itself is not simple - neither deriving it nor fine-tuning is intuitive, as can be said about other variations on the theme of gradient descent. The theme is simple: to reach a bottom of a hill, go in the steepest downhill direction[1]. Making it work - and making it work fast - requires modifications to this idea, which require quite a bit of work. [1]https://en.wikipedia.org/wiki/Gradient_descent https://en.wikipedia.org/wiki/Gradient_descent