3 ms·
Just an FYI but GPUs have been beefing up L2 caches over generations. Ada has a fat unified 96MB L2 cache smack in the middle of the crossbar.
by Veliladon 4y ago
Just an FYI but GPUs have been beefing up L2 caches over generations. Ada has a fat unified 96MB L2 cache smack in the middle of the crossbar.
- dragontamer 4y agoI'd say the per-core L2 cache of 2MB is a bigger deal for computations than a last-level-cache of size 96MB on the GPU that's shared between all cores. The per-core (or SM / WGP) L1 or L0 cache of GPUs is quite anemic. The GPU designers clearly "intend" for the programmer to hold as much state in registers as possible, rather than in cache, for their computations. You really want to keep the state at 1024 bytes or less in practice for high speed GPU computations. CPUs with 2MB L2 cache per core (alder lake from Intel), or 1MB cache per core (Apple M2 and/or AMD Zen4), are simply designed to handle bigger "states per thread". Using the full 1MB to 2MBs for this L2 cache is reasonable.