4 ms·

Here we are, its 2020, and intel's single core memory bandwidth numbers haven't really moved at all since nehalem. Despite the additional memory channels, and t

Here we are, its 2020, and intel's single core memory bandwidth numbers haven't really moved at all since nehalem. Despite the additional memory channels, and the increase in DDR bandwidth moving from DDR2 to DDR4 leading to machines with 100GB+/sec memory bandwidth (the large NUMA machines have significantly more). So, its pretty obvious they are choking off the single core bandwidth to keep a single core from starving the others in the machine. This 100% makes sense for server applications, but for workstation and desktop usage its completely crazy that a single core has 50-60 GB/sec L3 bandwidth, but can only utilize 10-20% of the available ram bandwidth to flush the L3 even when the other cores in the machine are idle. The ARM machine literally has somewhere between 2-4x the single core memory bandwidth, despite being in a configuration with more actual cores than intel offers.

BeeOnRope 6y ago
Yes, the number of L1 fill buffers was static at ~10 for almost a decade, and this, in combination with memory latency which didn't move much, proved the limit on single-core bandwidth. That said, in ICL there has been a nice jump. Graviton results to RAM certainly are nice. I've heard that in this type of store workload they can automatically use something like NT stores, avoiding the RFO (which would normally cut bandwidth in half). The Intel chips do get much better numbers if you use NT stores... but I didn't go into that since it wasn't the point of this article.
- StillBored 6y ago
  The NT vs vector ops/etc discussion shouldn't even happen on x86. A decade+ ago, Andy Glew expounded on the mistakes they made for the P6 when it came to the rep prefix. Apparently, they have finally fixed the startup times for it in icelake. But it continues to be a squandered opportunity because in theory they could have hidden a lot of the generational/microarch messiness behind the rep mov/sto sequences rather than requiring software updates every-time they updated the microarch like a RISC machine. AKA rep should be a lot harder to beat on any given machine. The nontemporal case is another one of these instances where the rep sequence could signal early on that the operation is going to be more efficient with a NT store. Reserving actual software defined NT stores for the cases where software is absolutely sure it won't fetch the data in the near future. There are a few other cases where high level microcoded instruction like this are an advantage because it allows a common piece of code to be implemented in differing ways depending on the actual product. Reference, Alpha PAL code.
fomine3 6y ago
Thank you for pointing this bottleneck out.

BeeOnRope 6y ago

Yes, the number of L1 fill buffers was static at ~10 for almost a decade, and this, in combination with memory latency which didn't move much, proved the limit on single-core bandwidth. That said, in ICL there has been a nice jump. Graviton results to RAM certainly are nice. I've heard that in this type of store workload they can automatically use something like NT stores, avoiding the RFO (which would normally cut bandwidth in half). The Intel chips do get much better numbers if you use NT stores... but I didn't go into that since it wasn't the point of this article.

StillBored 6y ago
The NT vs vector ops/etc discussion shouldn't even happen on x86. A decade+ ago, Andy Glew expounded on the mistakes they made for the P6 when it came to the rep prefix. Apparently, they have finally fixed the startup times for it in icelake. But it continues to be a squandered opportunity because in theory they could have hidden a lot of the generational/microarch messiness behind the rep mov/sto sequences rather than requiring software updates every-time they updated the microarch like a RISC machine. AKA rep should be a lot harder to beat on any given machine. The nontemporal case is another one of these instances where the rep sequence could signal early on that the operation is going to be more efficient with a NT store. Reserving actual software defined NT stores for the cases where software is absolutely sure it won't fetch the data in the near future. There are a few other cases where high level microcoded instruction like this are an advantage because it allows a common piece of code to be implemented in differing ways depending on the actual product. Reference, Alpha PAL code.

fomine3 6y ago

Thank you for pointing this bottleneck out.