3 ms·

Excellent article - writing NUMA-aware code such that it's aware of where a thread's memory is compared to what socket it's on is very important for high-perfor

by berkut 13y ago

Excellent article - writing NUMA-aware code such that it's aware of where a thread's memory is compared to what socket it's on is very important for high-performance code. You can get serious speed gains by reducing latency. Similarly, writing cache-aware code (in terms of the cache hierarchy) and locking down thread affinity can have serious benefits as well, especially on OSs with rather poor schedulers like OS X and Windows, which tend to bounce threads around a bit too much.

fulafel 13y ago
Multi-socket systems are uncommon these days. Tuning for NUMA can make a difference for the subset of users who can utilize more parallelism than you get from a single socket, but can't use a cluster (that would scale further than multi-socket system). Even then it only works if the app spends most of its time waiting for cache misses: it could reduce the runtime of a 60% stalled app by 20%, using the 100ns vs 150s latency figure in the article - assuming as the starting point that the OS heuristic memory placement fails pessimally. Less than 20% in more realistic cases. For the vast majority of apps, there are more serious perf gains to be had for same or less effort elsewhere.
- berkut 13y ago
  What? They're not uncommon for servers or high-performance workstations - Xeon dual/quad socket are very common. I've seen close to 250% speed increases with NUMA-aware code (and tying thread affinity down) - writing things so the code's aware of where the memory is and targets thread jobs to cores on sockets that have that memory already can significantly reduces memory traffic over the QPI links, which if you don't do (with AVX instructions trying to do 8 floats per clock with IB) systems tend to just be memory-starved, as they can't feed data to processors fast enough from main memory. Especially with Xeons, as the pre-fetchers are so aggressive.
  - fulafel 13y ago
    You moved the goalposts - I said they're uncommon in general. Though even most servers are single socket these days. Most high performance parallelized code runs in game engines, media codecs, and things like that. I can believe you can see speed increases like you cite with bandwidth-bound code, but would doubt this kind of optimization is good bang for the buck for most perf critical code.
    - berkut 13y ago
      What? Not if they're CPU-bound they're not. Look at Dell/HP workstation/servers: Z600/800 | Z620/Z820 range - dual sockets, all of them. The cheaper configs might only come with one CPU, but they've got two sockets. I'm talking highly-parallelisable code like in renders/raytracers, fluid simulations, image processing. All that stuff needs huge amounts of memory bandwidth. And it's generally parallisable in both threads and with SIMD with SSE/AVX, so you're literally up against the cache/memory throughput limits of the system. Game engines aren't that parallel actually - even job queue based ones which are in theory very scalable have dependencies which means that generally there's a limit to the number of threads they can run at once. The above examples I gave can generally scale linearly, given memory bandwidth.
      - fulafel 13y ago
        Most servers are doing pedestrian things like VMware, Exchange, web etc. and people buy server hardware for the RAS features. I'm saying apps and hardware you reference are uncommon. games was an example perf critical code that seldom benefits from numa optimization.
- kasey_junk 13y ago
  I haven't deployed a production application to a single socket server in over 4 years (and that was anomalous enough for me to remember it.) To say any thing for the "vast majority of apps" is a little out of bounds. In some cases on multi-socket machines a single cache miss takes as much time as ~1000 machine instructions. If you are working on calculation intensive operations that is a lot of overhead to pay for not having a good memory model.

by berkut 13y ago

fulafel 13y ago

Multi-socket systems are uncommon these days. Tuning for NUMA can make a difference for the subset of users who can utilize more parallelism than you get from a single socket, but can't use a cluster (that would scale further than multi-socket system). Even then it only works if the app spends most of its time waiting for cache misses: it could reduce the runtime of a 60% stalled app by 20%, using the 100ns vs 150s latency figure in the article - assuming as the starting point that the OS heuristic memory placement fails pessimally. Less than 20% in more realistic cases. For the vast majority of apps, there are more serious perf gains to be had for same or less effort elsewhere.

berkut 13y ago
What? They're not uncommon for servers or high-performance workstations - Xeon dual/quad socket are very common. I've seen close to 250% speed increases with NUMA-aware code (and tying thread affinity down) - writing things so the code's aware of where the memory is and targets thread jobs to cores on sockets that have that memory already can significantly reduces memory traffic over the QPI links, which if you don't do (with AVX instructions trying to do 8 floats per clock with IB) systems tend to just be memory-starved, as they can't feed data to processors fast enough from main memory. Especially with Xeons, as the pre-fetchers are so aggressive.
- fulafel 13y ago
  You moved the goalposts - I said they're uncommon in general. Though even most servers are single socket these days. Most high performance parallelized code runs in game engines, media codecs, and things like that. I can believe you can see speed increases like you cite with bandwidth-bound code, but would doubt this kind of optimization is good bang for the buck for most perf critical code.
  - berkut 13y ago
    What? Not if they're CPU-bound they're not. Look at Dell/HP workstation/servers: Z600/800 | Z620/Z820 range - dual sockets, all of them. The cheaper configs might only come with one CPU, but they've got two sockets. I'm talking highly-parallelisable code like in renders/raytracers, fluid simulations, image processing. All that stuff needs huge amounts of memory bandwidth. And it's generally parallisable in both threads and with SIMD with SSE/AVX, so you're literally up against the cache/memory throughput limits of the system. Game engines aren't that parallel actually - even job queue based ones which are in theory very scalable have dependencies which means that generally there's a limit to the number of threads they can run at once. The above examples I gave can generally scale linearly, given memory bandwidth.
    - fulafel 13y ago
      Most servers are doing pedestrian things like VMware, Exchange, web etc. and people buy server hardware for the RAS features. I'm saying apps and hardware you reference are uncommon. games was an example perf critical code that seldom benefits from numa optimization.
kasey_junk 13y ago
I haven't deployed a production application to a single socket server in over 4 years (and that was anomalous enough for me to remember it.) To say any thing for the "vast majority of apps" is a little out of bounds. In some cases on multi-socket machines a single cache miss takes as much time as ~1000 machine instructions. If you are working on calculation intensive operations that is a lot of overhead to pay for not having a good memory model.