2 ms·
Memory and caching is a big part of nondeterminism, but it's not just about memory. GPUs employ a far more primitive execution strategy than modern server CPUs
by pron 12y ago
Memory and caching is a big part of nondeterminism, but it's not just about memory. GPUs employ a far more primitive execution strategy than modern server CPUs with their branch prediction, ILP etc. Also, GPUs, while terrific for parallel workloads, are terrible for concurrent workloads, which require very intricate branching.
- seanmcdirmid 12y agoWell, you can branch as much as you want on the GPU, as long as your branches are coherent among each thread sharing the same control unit :) I wouldn't say this is "primitive", just a very different way in thinking about computation that has been very very very effective for a lot of use cases. GPUs are obviously not the solution for concurrent or irregular workloads (yet), but many are surprised how much mileage one can get out of Python + CUDA for scientific workloads. The only point I was making is that there is a world where the hardware is much more deterministic (even if most of us can't go there).