3 ms·

In the vast majority of cases, people using ODE solvers are working with physical systems, where there likely aren't more than a dozen digits of accuracy in any

In the vast majority of cases, people using ODE solvers are working with physical systems, where there likely aren't more than a dozen digits of accuracy in any of the parameters. So you set some fixed level of accuracy for the output, and seek a method that can reliably attain that accuracy at the lowest cost. (Either that, or the system is too chaotic, in which case you're told not to bother since the output would be meaningless anyway.) Something I've had trouble finding any information about is, what if we have a mathematical system, where the initial configuration is known to infinite precision, and we want to solve the ODE to an arbitrary precision (say, 100 digits, or 1000 digits)? All the usual stepwise methods would seemingly require a very high order to avoid a galactic step count, but initializing all the coefficients, etc., would be a struggle in itself. Is this the best that can be done, or are there less-usual methods that would win out at extremely high accuracy levels?

kronicum2025 1y ago
It depends on your dimensions of the system. If your system is one or low dimensional or if the function has sufficient smoothness, then it makes sense to assume that the function is a series of some kind, and then iterative solve on the least squares using gradient descent. If you pick the correct kind of series, you would get extremely quick convergence. When number of dimensions become very high or if the function becomes extremely irregular, then variations of monte carlo are generally extremely better than any other methods and have much higher accuracy than other methods, but the accuracy is still much lower than low dimensional methods.
- LegionMammal978 1y ago
  > It depends on your dimensions of the system. If your system is one or low dimensional or if the function has sufficient smoothness, then it makes sense to assume that the function is a series of some kind, and then iterative solve on the least squares using gradient descent. If you pick the correct kind of series, you would get extremely quick convergence. Thank you, this sounds like what I'm looking for. Would you know of any further resources on this? Most of what I've been playing with has 4 dimensions or fewer. (E.g., in one particular problem that vexed me, I had a 2D vector x(t) and an ODE x'(t) = F(t,x(t)), where F is smooth and well-behaved except for a singularity at the origin x = (0,0). It always hits the origin in finite time, so I wanted to calculate that hitting time from a given x(0). Series solutions work well in the vicinity of the origin, the problem is accurately getting to that vicinity.)
  - kronicum2025 1y ago
    In such cases, you do need to switch between methods. If you already know how to deal with origin, then getting to origin can be done with a normal stepper. Decide how low you are okay with your step to go. Set that as your target `t_target` which would typically be a very small number. Approach that number with normal stepping methods and then once you are there, start your series solve.
adgjlsfhk1 1y ago
there are adaptive order ODE solvers that can adapt to arbitrarily high order. For example, e.g. https://arxiv.org/abs/2412.14362 https://arxiv.org/abs/2412.14362 (which I'm a co-author on) can get to ~200th order which should be enough to efficiently solve an ODE into 10s of thousands of digits.
krull10 1y ago
Use spectral methods? But you’d presumably also have to use higher precision number types if you want that many digits.
- LegionMammal978 1y ago
  Of course you'd have to use arbitrary-precision arithmetic, that's the easy part. The hard part is knowing what to do with it, if you're someone like me who isn't an expert in ODEs. Do you know of any accessible resources on spectral methods? (Of course arbitrary precision is far, far worse than double precision in terms of performance, which is why so much work goes into techniques that avoid increasing precision. But sometimes you have few enough instances to solve that you can simply eat the cost.)

kronicum2025 1y ago

It depends on your dimensions of the system. If your system is one or low dimensional or if the function has sufficient smoothness, then it makes sense to assume that the function is a series of some kind, and then iterative solve on the least squares using gradient descent. If you pick the correct kind of series, you would get extremely quick convergence. When number of dimensions become very high or if the function becomes extremely irregular, then variations of monte carlo are generally extremely better than any other methods and have much higher accuracy than other methods, but the accuracy is still much lower than low dimensional methods.

LegionMammal978 1y ago
> It depends on your dimensions of the system. If your system is one or low dimensional or if the function has sufficient smoothness, then it makes sense to assume that the function is a series of some kind, and then iterative solve on the least squares using gradient descent. If you pick the correct kind of series, you would get extremely quick convergence. Thank you, this sounds like what I'm looking for. Would you know of any further resources on this? Most of what I've been playing with has 4 dimensions or fewer. (E.g., in one particular problem that vexed me, I had a 2D vector x(t) and an ODE x'(t) = F(t,x(t)), where F is smooth and well-behaved except for a singularity at the origin x = (0,0). It always hits the origin in finite time, so I wanted to calculate that hitting time from a given x(0). Series solutions work well in the vicinity of the origin, the problem is accurately getting to that vicinity.)
- kronicum2025 1y ago
  In such cases, you do need to switch between methods. If you already know how to deal with origin, then getting to origin can be done with a normal stepper. Decide how low you are okay with your step to go. Set that as your target `t_target` which would typically be a very small number. Approach that number with normal stepping methods and then once you are there, start your series solve.

adgjlsfhk1 1y ago

there are adaptive order ODE solvers that can adapt to arbitrarily high order. For example, e.g. https://arxiv.org/abs/2412.14362 https://arxiv.org/abs/2412.14362 (which I'm a co-author on) can get to ~200th order which should be enough to efficiently solve an ODE into 10s of thousands of digits.

krull10 1y ago

Use spectral methods? But you’d presumably also have to use higher precision number types if you want that many digits.

LegionMammal978 1y ago
Of course you'd have to use arbitrary-precision arithmetic, that's the easy part. The hard part is knowing what to do with it, if you're someone like me who isn't an expert in ODEs. Do you know of any accessible resources on spectral methods? (Of course arbitrary precision is far, far worse than double precision in terms of performance, which is why so much work goes into techniques that avoid increasing precision. But sometimes you have few enough instances to solve that you can simply eat the cost.)