5 ms·

Came to say exactly this. A lot of code is wasting a ton of performance on servers because of not being NUMA aware. We should also test high throughput/perform

Came to say exactly this. A lot of code is wasting a ton of performance on servers because of not being NUMA aware. We should also test high throughput/performance code on NUMA systems. Remember to allocate local memory for your threads — IOW, be NUMA aware. Going through interconnect(s) is slow.

jjoonathan 5y ago
How is that generally done? Platform specific libraries? Utilities that ask the OS to pin a program to a node?
- pkaye 5y ago
  In Linux you can use numactl command to pin a process.
- cyber_kinetist 5y ago
  libnuma probably does the job. https://github.com/numactl/numactl https://github.com/numactl/numactl
- magicalhippo 5y ago
  Windows has a NUMA API[1] to help with these things. For pinning a thread you can for example use the SetThreadSelectedCpuSetMasks call[2] from what I can gather. [1]: https://docs.microsoft.com/en-us/windows/win32/procthread/numa-support https://docs.microsoft.com/en-us/windows/win32/procthread/nu... [2]: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-setthreadselectedcpusetmasks https://docs.microsoft.com/en-us/windows/win32/api/processth...
- profquail 5y ago
  If you want to do set thread-core affinity in a cross-platform way, hwloc is a good approach: https://www.open-mpi.org/projects/hwloc/doc/v2.7.0/a00166.php#gae42c01b2addcfbf6048b9a516dd7a906 https://www.open-mpi.org/projects/hwloc/doc/v2.7.0/a00166.ph...
hardolaf 5y ago
I'm in HFT and a huge part of our job for the FPGA drivers is managing the NUMA awareness of our software. And we have to constantly educate new devs coming into the low level software teams about this. Luckily, the SW teams are starting to do this training for us but it's always a major headache when there's issues. You can easily lose 50%+ of your performance by just failing to set a core affinity without even considering anything else.

jjoonathan 5y ago

How is that generally done? Platform specific libraries? Utilities that ask the OS to pin a program to a node?

pkaye 5y ago
In Linux you can use numactl command to pin a process.
cyber_kinetist 5y ago
libnuma probably does the job. https://github.com/numactl/numactl https://github.com/numactl/numactl
magicalhippo 5y ago
Windows has a NUMA API[1] to help with these things. For pinning a thread you can for example use the SetThreadSelectedCpuSetMasks call[2] from what I can gather. [1]: https://docs.microsoft.com/en-us/windows/win32/procthread/numa-support https://docs.microsoft.com/en-us/windows/win32/procthread/nu... [2]: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-setthreadselectedcpusetmasks https://docs.microsoft.com/en-us/windows/win32/api/processth...
profquail 5y ago
If you want to do set thread-core affinity in a cross-platform way, hwloc is a good approach: https://www.open-mpi.org/projects/hwloc/doc/v2.7.0/a00166.php#gae42c01b2addcfbf6048b9a516dd7a906 https://www.open-mpi.org/projects/hwloc/doc/v2.7.0/a00166.ph...

hardolaf 5y ago

I'm in HFT and a huge part of our job for the FPGA drivers is managing the NUMA awareness of our software. And we have to constantly educate new devs coming into the low level software teams about this. Luckily, the SW teams are starting to do this training for us but it's always a major headache when there's issues. You can easily lose 50%+ of your performance by just failing to set a core affinity without even considering anything else.