I'd say the per-core L2 cache of 2MB is a bigger deal for computations than a last-level-cache of size 96MB on the GPU that's shared between all cores. The per

I'd say the per-core L2 cache of 2MB is a bigger deal for computations than a last-level-cache of size 96MB on the GPU that's shared between all cores. The per-core (or SM / WGP) L1 or L0 cache of GPUs is quite anemic. The GPU designers clearly "intend" for the programmer to hold as much state in registers as possible, rather than in cache, for their computations. You really want to keep the state at 1024 bytes or less in practice for high speed GPU computations. CPUs with 2MB L2 cache per core (alder lake from Intel), or 1MB cache per core (Apple M2 and/or AMD Zen4), are simply designed to handle bigger "states per thread". Using the full 1MB to 2MBs for this L2 cache is reasonable.