2 ms·
> Their inverted cache hierarchy and lack of persistence between kernel launches is a concern for memory locality. Hmm, I can certainly agree to that. There's
by dragontamer 6y ago
> Their inverted cache hierarchy and lack of persistence between kernel launches is a concern for memory locality.
Hmm, I can certainly agree to that. There's some tricks to try to minimize that issue, but they're complicated to use and really screw up the overall architecture.
Batching up a larger batch of things to process can help for that. But GPUs have much smaller caches in general and are clearly designed for running out of VRAM. (With caches mainly for atomic / synchronization here and there). A100 only has 40MBs of L2 cache (last level), but the extra threads really eat up that space faster than you expect.