> Can you give concrete examples when data processing on the CPU is clearly better than on the GPU? Pretty much anything where the matmul is important to perfo

> Can you give concrete examples when data processing on the CPU is clearly better than on the GPU? Pretty much anything where the matmul is important to performance, but doesn't dominate it and requires serial steps. A classic example off the top of my head would be solving a matrix differential equation[1]. Here's an example in Julia using DifferentialEquations.jl and CUDA.jl for the GPU part: This is solving the differential equation du/dt = A * u for the cases where u and A are arrays on the GPU versus when they are arrays on the CPU. I'm doing the solving in-place to try and minimize the amount of expensive allocations. using DifferentialEquations, CUDA function mysolve(u0, A, tspan) f!(du, u, A, t) = mul!(du, A, u) prob = ODEProblem(f!, u0, tspan, A) solve(prob) end mysolve (generic function with 1 method) julia> let n = 50, u0 = rand(Float32, n), A = randn(Float32, n, n) # Make GPU versions of u0 and A cuu0 = cu(u0) cuA = cu(A) # Do a run of the code so there's no compiler latency biasing the results mysolve( u0, A, (0.0, 1.0)) mysolve(cuu0, cuA, (0.0, 1.0)) # Now run the compiler code and time it's execution: @time mysolve( u0, A, (0.0, 1.0)) @time mysolve(cuu0, cuA, (0.0, 1.0)) end; 0.000314 seconds (621 allocations: 98.281 KiB) 0.007218 seconds (10.33 k allocations: 382.266 KiB) In this case, the CPU code was ~20x faster than the GPU code. True, I could get a better GPU (I'm using a RTX 2060), but I could also get a better CPU (Ryzen 5 2600). There's not much point in using a GPU for this example, and I think that just comes down to the fact that there's a non-trivial amount of serial work that needs to be done between the matmuls. [1] https://en.wikipedia.org/wiki/Matrix_differential_equation https://en.wikipedia.org/wiki/Matrix_differential_equation