Right, I hate to be that guy, but it essentially boils down to "take a random step and see if it's good, if so update" (with some fanciness). It should go witho

Right, I hate to be that guy, but it essentially boils down to "take a random step and see if it's good, if so update" (with some fanciness). It should go without saying that this is horribly inefficient in a very high dimensional space (e.g. the number of parameters). The results are decent, but it's just another in a long line of work (and more recent work https://arxiv.org/abs/2202.08587 https://arxiv.org/abs/2202.08587 https://openreview.net/pdf?id=5i7lJLuhTm https://openreview.net/pdf?id=5i7lJLuhTm) trying to use backprop algorithm with some promising results but still falling far short of the standard backprop workhorse.