4 ms·
Why not? It's not like moving it to the same chip would make DRAM perform better, the signal lines are not the limiting factor there.
by stefan_ 8y ago
Why not? It's not like moving it to the same chip would make DRAM perform better, the signal lines are not the limiting factor there.
- wmf 8y agoHBM2 does have dramatically more bandwidth than DDR4.
- monocasa 8y agoIt has more bandwidth but about the same latency. CPUs fundamentally attack memory latency differently than GPUs do, and raw bandwidth is less important.
- imtringued 8y agoIt is extremely important when you have 64 cores orif you use wide SIMD instructions. Intel's Xeon Phi have 16GB HMC memory with 500GB/s bandwidth to prevent the CPUs from being starved by main memory. Imagine you have a cache miss on literally every memory access and have to pay a 100 nano second penalty to access main memory. SMT allows the CPU to switch to a different thread during a cache miss. Effectively you can have 128 memory loads at any given time. 100ns amortized over 128 threads is less than 1 nano second. Since the minimum size you can read is a 64 byte cache line this means that program will require at least 64GB/s of memory bandwidth even though your CPU is stuck on cache misses.
- monocasa 8y agoThat's what I meant by "CPUs fundamentally attack memory latency differently than GPUs do"; they rely on their relatively smart caches and prefetching logic to reduce bandwidth explosion from all of the cores, and the latency of accessing external DRAM. Of course if you thrash your cache on a modern general purpose CPU, you're going to have a really bad time. Pretty much if you're getting close to worse case memory access like in your example, you need to step back and re evaluate quite a few of your choices.