This is a common confusion. Both OP and GP are talking about two different simplex methods. Simplex refers to two algorithms: [1] Simplex method for solving LPs

This is a common confusion. Both OP and GP are talking about two different simplex methods. Simplex refers to two algorithms: [1] Simplex method for solving LPs, initially proposed by George Dantzig. [2] Nelder-Mead Simplex method, which is a derivative-free algorithm for nonlinear functions. The article refers to [1] whereas the GP probably referred to [2]. Nelder-Mead simplex only requires function evaluations, which makes it amenable to situations where the derivative evaluations are impossible or the function evaluations are expensive (like a simulation). BFGS approximates the Hessian (2nd order derivative) and will generally outperform Nelder Mead unless derivatives are not available or if the terrain has lots of local optima or saddle points. [1] https://en.wikipedia.org/wiki/Simplex_algorithm https://en.wikipedia.org/wiki/Simplex_algorithm [2] https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method