3 ms·
In GPUs, in my opinion, having laggy cache is nowhere near as scary as in CPU. GPU caches are said to be all about prefetch logic.
by baybal2 8y ago
In GPUs, in my opinion, having laggy cache is nowhere near as scary as in CPU. GPU caches are said to be all about prefetch logic.
- dragontamer 8y agoI agree in principle, but the SLM isn't a "cache", at least the equivalent of it in AMD and NVidia GPUs. AMD LDS and NVidia Shared Memory is extremely fast. Best-case scenario throughput of one AMD LDS operation every 2-clock cycles. That's 64-threads performing a 32-bit load or store operation (64x32-bit load/store operations) in just 2-clock cycles (assuming you have no bank-conflicts / ideal situations)... per AMD execution unit. You ain't got nothing on the power of this gather/scatter to AMD LDS. We're talking ~TBps bandwidth here, not GBps... with incredibly low latency guarantees. My understanding is that NVidia would perform this operation at a similar speed to its shared memory (under similar "no bank conflicts" situations) per SM. NVidia shares their "shared memory" with L1 cache (so yeah, the architecture is different than AMD), but its performance is also huge and measured in ~TBps. Note that this gather/scatter is synchronized between all 64-threads on a AMD workgroup, or all 32-threads of a NVidia warp. As such, you have ~2-cycles of latency between thread communications in the ideal case. As such, SLM isn't "cache". Its the simplest (and 2nd fastest way) to share data between threads in a workgroup (there are harder-to-use "shuffle" intrinsics available, if you have static thread communications, such as those found in a sorting network... or in a "reduce" operation). And Intel's implementation of it in Gen9 was far slower than its competitors. Intel is at least claiming that they fixed this problem in Gen11. I'm cautiously optimistic: 3rd party benchmarks on Gen11 iGPUs would be nice to confirm if they really sped up SLM as much as it looks like. --------- EDIT: Now that I think of it, maybe you were replying to my Point#3. At which point... yeah, I agree with you. Its hard to tell what part of my post you were addressing though.
- baybal2 8y agoI was thinking that cache lookups were not their stronger side as they had fancy 4+ level cache. This is what I would've expected in a regular arm soc. The more complex is the cache architecture, the more laggy your main ram should be to mandate that. That's what I thought.