I cannot find any such numbers in the paper. What the paper says is that MeZO converges much slower than SGD, and each step needs two forward passes. "As a lim

I cannot find any such numbers in the paper. What the paper says is that MeZO converges much slower than SGD, and each step needs two forward passes. "As a limitation, MeZO takes many steps in order to achieve strong performance."