Occationally, there are a few places where you want arbitary high derivatives. But in general there are specialized solutions for those (though very few things

Occationally, there are a few places where you want arbitary high derivatives. But in general there are specialized solutions for those (though very few things implement them. E.g. Jax's JETs and TaylorSeries.jl in julia implement taylor-mode AD. But those also need custom primatives) More of a problem is first derivatives, but that you need very high accuracy. Which doesn't occur in ML but does occur sometimes in scientific computing. (Doesn't occur in ML cos we just basically shake the thing about a bit anyway)