3 ms·
You're still quoting clock speeds. That's not how this works. Go check a timing diagram for a DRAM cycle in your part of choice and do the math.
by ajross 6y ago
You're still quoting clock speeds. That's not how this works. Go check a timing diagram for a DRAM cycle in your part of choice and do the math.
- dragontamer 6y agoDo you know what 3200MHz / PC4-25600 DDR4 means? 25600 is the channel rate in (EDIT) MB/sec of the stick of RAM. That's 25GB/s for a 3200 MHz DDR4 stick. x8 (for 8-channels working in parallel) is 200GB/s. ----------- This has been measured in practice by Netflix: https://2019.eurobsdcon.org/slides/NUMA%20Optimizations%20in%20the%20FreeBSD%20Network%20Stack%20-%20Drew%20Gallatin.pdf https://2019.eurobsdcon.org/slides/NUMA%20Optimizations%20in... As you can see, Netflix's FreeBSD optimizations have allowed EPYC to reach 194GB/s measured performance (or just under the 200GB/s theoretical). And only with VERY careful NUMA-tuning and extreme optimizations were they able to get there.
- gbl08ma 6y agoAll of that is bandwidth and clock speed, not latency
- dragontamer 6y agoLook, if CPUs were better at memory latency, the BVH-traversal of raytracing would still be done on CPUs. BVH-tree traversals are done on the GPU now for a reason. GPUs are better at latency hiding and taking advantage of larger sets of bandwidth than CPUs. Yes, even on things like pointer-chasing through a BVH-tree for AABB bounds checking. GPUs have pushed latency down and latency-hiding up to unimaginable figures. In terms of absolute latency, you're right, GPUs are still higher latency than CPUs. But in terms of "practical" effects (once accounting for latency hiding tricks on the GPU, such as 8x way occupancy (similar to hyperthreading), as well as some dedicated datastructures / programming tricks (largely taking advantage of the millions of rays processed in parallel per frame), it turns out that you can convert many latency-bound problems into bandwidth-constrained problems. ----------- That's the funny thing about computer science. It turns out that with enough RAM and enough parallelism, you can convert ANY latency-bound problem into a bandwidth-bound problem. You just need enough cache to hold the results in the meantime, while you process other stuff in parallel. Raytracing is an excellent example of this form of latency hiding. Bouncing a ray off of your global data-structure of objects involved traversing pointers down the BVH tree. A ton of linked-list like current_node = current_node->next like operations (depending on which current_node->child the ray hit). From the perspective of any ray, it looks like its latency-bound. But from the perspective of processing 2.073 million rays across a 1920 x 1080 video game scene with realtime-raytracing enabled, its bandwidth bound.
- wmf 6y agoThat presentation shows 194 gigabits/s which is only ~24 gigabytes/s at the NIC; that requires ~96 gigabytes/s of memory bandwidth. Usable memory bandwidth on Milan is only <120 gigabytes/s which is about 60% of the theoretical max. DRAM never gets more than ~80% of theoretical max bandwidth because of command overhead (which is what I think ajross keeps alluding to). https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scalable-review/4 https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scal...
- dragontamer 6y agoI appreciate the correction. It seems like I made the mistake of Gbit vs GByte confusion (little-b vs big-B). > (which is what I think ajross keeps alluding to) It seems like ajross is accusing me of underestimating CPU-bandwidth. At least, that's my interpretation of the discussion so far. As you've pointed out however, I'm overestimating it. EDIT: But I'm overestimating it on both sides. A100 2000 TB/s is the "channel bandwidth" as well, as the CAS and RAS commands still need to go through the channel and get interpreted.