3 ms·
The original Neural ODEs paper is quite readable, and by now there are loads of blog posts and even a few talks on the subject. The basic idea is inspired by t
by ssivark 5y ago
The original Neural ODEs paper is quite readable, and by now there are loads of blog posts and even a few talks on the subject.
The basic idea is inspired by the “adjoint method” for ODE solving (so you don’t have to hold in memory all the intermediate layer outputs — which is otherwise necessary to compute the backpropagated gradient signal).
- ChrisRackauckas 5y agoYeah, though with the method described in that paper you do have to be very careful since it has exponential error growth with the Lipchitz constants of the ODE. See https://aip.scitation.org/doi/10.1063/5.0060697 https://aip.scitation.org/doi/10.1063/5.0060697 for details. But that is generally the case in numerical analysis: there's always a simple way to do things, and then there's the way that prevents error growths. Both have different pros and cons.