It really depends on where you're looking. We tested neural ODE compatible solver benchmarks from 3 ODEs and about 1 million ODEs and saw an advantage for Julia

It really depends on where you're looking. We tested neural ODE compatible solver benchmarks from 3 ODEs and about 1 million ODEs and saw an advantage for Julia of >30x across the board [1]. Then showed the spiral neural ODE from the original paper train 100x faster in Julia [2]. So it should at least be qualified as, PyTorch has some good kernels, and it has some things it doesn't optimize close to as much as Julia. [1] https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320 https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e... [2] https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37da837bfc32c4 https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37...