DFO is far slower to converge in general than gradient based methods even when it’s close to the optimum because it doesn’t use derivative information. Newton’s

DFO is far slower to converge in general than gradient based methods even when it’s close to the optimum because it doesn’t use derivative information. Newton’s method on the other hand has local quadratic convergence. Not sure I would bet on DFOs to have better performance than gradient methods. DFOs have two advantages: they’re embarrassingly parallelizable and for non convex problems, they don’t succumb as easily to local optima. But they’re not better performing in general.