3 ms·

With CPUs the bottleneck isn't just memory bandwidth only but also memory latency. If prefetch fails to get the correct data loaded into the cache, the CPU just

With CPUs the bottleneck isn't just memory bandwidth only but also memory latency. If prefetch fails to get the correct data loaded into the cache, the CPU just stalls for thousands of cycles waiting for RAM to deliver the data. If I'm correct, this is the difference between DDR RAM and GDDR RAM - the latter has higher bandwidth but also higher latency, which is great for graphics with its high bandwidth requirements and very predictable memory access patterns (except when you do ray tracing) which allows you to hide the latency, but you don't want to use GDDR for system RAM if you value performance.

Const-me 3y ago
Yeah, the throughput/latency tradeoff is real. Here’s an unusual motherboard salvaged from Xbox Series X which combines AMD Zen2 CPU cores, GDDR6 memory, where the integrated GPU is disabled due to defects: https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-why-cpus-do-not-come-with-gddr6 https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-w... The performance is not great. GPUs are good at hiding the latency due to high degree of parallelism. GPU cores switch to another thread instead of waiting for the data. Each CPU core only runs 2 threads, it can’t quite do the same thing. GPU memory access patterns are only great for well written compute shaders (or CUDA kernels) when the developer who wrote that shader managed to do fully coalesced loads and stores. Memory access patterns for pixel shaders are not that great, especially when doing trilinear or anisotropic sampling in these shaders.
HappyDaoDude 3y ago
Which is why I always found it funny to see consoles using unified GDDR for CPU and GPU. But you can optimize around the limitation (within reason) and don't have to content with a general non-specified OS. This is why I do wonder about the path not taken with the 1T-SRAM that powered the Gamecube/Wii. It was designed purely with low latency in mind but I don't think it scaled well.

Const-me 3y ago

Yeah, the throughput/latency tradeoff is real. Here’s an unusual motherboard salvaged from Xbox Series X which combines AMD Zen2 CPU cores, GDDR6 memory, where the integrated GPU is disabled due to defects: https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-why-cpus-do-not-come-with-gddr6 https://www.tomshardware.com/news/4800s-xbox-chip-shows-us-w... The performance is not great. GPUs are good at hiding the latency due to high degree of parallelism. GPU cores switch to another thread instead of waiting for the data. Each CPU core only runs 2 threads, it can’t quite do the same thing. GPU memory access patterns are only great for well written compute shaders (or CUDA kernels) when the developer who wrote that shader managed to do fully coalesced loads and stores. Memory access patterns for pixel shaders are not that great, especially when doing trilinear or anisotropic sampling in these shaders.

HappyDaoDude 3y ago

Which is why I always found it funny to see consoles using unified GDDR for CPU and GPU. But you can optimize around the limitation (within reason) and don't have to content with a general non-specified OS. This is why I do wonder about the path not taken with the 1T-SRAM that powered the Gamecube/Wii. It was designed purely with low latency in mind but I don't think it scaled well.