I am not familiar with these algorithms, but: Many common functions have non-derivative friendly discontinuities. Logic functions, rounding, conditional clause

I am not familiar with these algorithms, but: Many common functions have non-derivative friendly discontinuities. Logic functions, rounding, conditional clauses, sawtooth functions, etc.) Other non-derivative functions have flat constant zero-derivative areas. Like logic functions & rounding (again), etc. Some functions have derivatives but have so many minima (or maxima) that non-optimal or non-acceptable minima are likely to trap learning. Sine and cosine waves, for instance. Some functions even have repeating discontinuities going through +/-infinity. Tangent and secant. Or, in some cases a model may exist of the system (or part of the system) to be optimized, which can be used to calculate the system response, but it contains relationships that are not easily accessible, so accurate derivatives are difficult or time costly to estimate. Since there is no symbolic expression to allow for efficient calculation of the derivatives.