Sometimes, yes. Other times, no. If you're using an explicit integrator like a Runge–Kutta method such as RK4, then the answer is yes and the algorithms map p

Sometimes, yes. Other times, no. If you're using an explicit integrator like a Runge–Kutta method such as RK4, then the answer is yes and the algorithms map pretty well onto a GPU. In that way, GPUs have been a huge boon to the scientific computing world. Though, personally, I find them a pain to program. However, if you use an implicit integrator like a backward Euler method, the answer is no because we need to solve a linear system. Yes, if we have a well defined problem it may be possible to design a custom preconditioner based on the physics of the problem that maps really well to a GPU. Those preconditioners take a huge amount of work to develop and, honestly, if you look at the last step of something like a multilevel method, it comes down to a direct factorization. Basically, direct dense and sparse factorizations do not currently map well to a GPU and these factorizations are extremely important to many scientific problems. Specifically, implicit time integrators depend on them to solve the linear systems involved and we need implicit integrators for stiff systems. Outside of integration, there are other situations where GPUs don't work well. In large scale optimization with equality constraints, there are linear systems that need to be solved and, most of the time, we need a direct factorization to solve these systems. That's why reduced spaced methods for things like parameter estimation are so popular. Basically, they're null space algorithms that eliminate the equality constraints by doing an implicit linear system solve using an explicit time integrator, which can map to something like a GPU. This leads to a host of other problems, but at least we can scale, sort of, sometimes. At the end of the day, most scientific problems based on continuum mechanics need really fast level 1, 2, and 3 BLAS operations. That's mostly enough for simulating physics based on explicit time integrators. For elliptic problems or problems that require implicit time integrators, we need factorizations. Most of the time, we can get away with LU, Choleski, QR, and SVD. Both dense and sparse are required. For optimization with equality constraints, factorizations are also required. By the way, if anyone wants to figure out how to do faster factorizations, dense and sparse, on whatever new hardware is coming out, that'd have an enormous impact on the scientific community. There are people working on it. There's not a lot of them.