4 ms·

Thanks for the thoughtful comment, pretty fascinating stuff. > In today's x86_64 hardware, there's no "supercomputer memory subsystem". It's just a glorified N

Thanks for the thoughtful comment, pretty fascinating stuff. > In today's x86_64 hardware, there's no "supercomputer memory subsystem". It's just a glorified NUMA system, and the biggest problem is putting the memory close to your core, i.e. keeping data local in your NUMA node to reduce latencies. I mean, memory topology varies greatly by uarch (doubly so between vendors). I can't take a routine tuned to Nehalem, run it on Haswell or Skylake and expect it to stay competitive. More generally, different hardware has different bandwidth and latency ratios, which affects software design (e.g. software written for commodity Dell w/ PCIe cards probably won't translate to Cray accelerator grid connected by HPE slingshot). And then there's hardware-specific features like RNICs bypassing DRAM and writing RDMA messages directly into the receiver's cache. So I think that ccNUMA and data locality is not sufficient to reason about memory perf.

bayindirh 3y ago
> I mean, memory topology varies greatly by uarch... You're absolutely right, this is why I said that if you're using libraries, this burden is generally handled by them. Also compilers do this and handle this very well. If you're writing your own routines, the best way is to read the arch docs, maybe some low-level sites like chips and cheese, do some synthetic benchmarks and write your code in a semi informed way. After writing the code, a suite of cachegrind, callgrind and perf is on order. See if there are any other bottlenecks, and tune your code accordingly. Add hints for your compiler, if possible. I was able to reach insane saturation levels with Eigen plus, some hand-tuned code. For the next level, I needed to change my matrix ordering, but it was already fast enough (30 minutes to 45 seconds: 40x speedup), so I left it there. Sometimes there are no replacement for blood, sweat and tears in this thing. I have never played with custom interconnects (Slingshot, etc.), yet, so I can't tell much.

bayindirh 3y ago

> I mean, memory topology varies greatly by uarch... You're absolutely right, this is why I said that if you're using libraries, this burden is generally handled by them. Also compilers do this and handle this very well. If you're writing your own routines, the best way is to read the arch docs, maybe some low-level sites like chips and cheese, do some synthetic benchmarks and write your code in a semi informed way. After writing the code, a suite of cachegrind, callgrind and perf is on order. See if there are any other bottlenecks, and tune your code accordingly. Add hints for your compiler, if possible. I was able to reach insane saturation levels with Eigen plus, some hand-tuned code. For the next level, I needed to change my matrix ordering, but it was already fast enough (30 minutes to 45 seconds: 40x speedup), so I left it there. Sometimes there are no replacement for blood, sweat and tears in this thing. I have never played with custom interconnects (Slingshot, etc.), yet, so I can't tell much.