You guys are talking past each other but really talking about the same thing - arithmetic intensity. You're talking about FEA or some other grid solver/discreti

You guys are talking past each other but really talking about the same thing - arithmetic intensity. You're talking about FEA or some other grid solver/discretized PDE/DFT type thing where the matmuls are small because the mesh is highly refined and you've assumed the potentials/fields/effects are hyper-local. But that's not accident or dumb luck - the problems in scientific HPC are modeled using these kinds potentials post-hoc ie so that they can be distributed across so many cores. What I'm saying is it's not like a global solver (ie taking into account all to all interactions) wouldn't be more accurate right? It's just an insane proposition because surprise surprise that would require an enormous matmul during the update, which you can't do efficiently, even on a GPU, for the same reason the ML folks can't: the arithmetic intensity isn't high enough and so you can incur i/o costs (memory or network, same thing at this scale).