4 ms·
The modern world of autodiff actually makes real second and third order derivatives fairly cheap to compute. The stuff that is actually used most commonly, onl
by snovv_crash 11mo ago
The modern world of autodiff actually makes real second and third order derivatives fairly cheap to compute.
The stuff that is actually used most commonly, only uses first order derivatives though (gradient descent, Levenberg-Marquardt, Kalman filters...)
- hansvm 11mo agoI could be wrong, but "memory" and "patience" sounded like they were referring to machine memory and patience waiting for a slow algorithm, which is what you would expect from any derivative more involved than a Jacobian for nontrivial problems, even when doing tricks like vjp or vhp.