4 ms·

in C++ std::shared_ptr (similar story) has similar effect. One of our applications (3D editor) went way slower when an artist were given a server-class machine

by malkia 3y ago

in C++ std::shared_ptr (similar story) has similar effect. One of our applications (3D editor) went way slower when an artist were given a server-class machine (NUMA) and we had to ensure that all threads would run on a single CPU socket (yes they were still accessing the "other" memory, but it was better somehow).

BazookaMusic 3y ago
I might be wrong on this explanation, but the reason why it was faster might have been the following: During execution you had two kinds of memory locations, some in CPU caches and some in RAM. By running all the threads on one socket, everything accessed from the cache was just a fast cache access. Everything accessed from the memory was a slower memory load. Frequently loaded/stored locations will tend to go to the cache. In the NUMA setup, you would have a larger cache (more than one socket) which would mean that more locations were likely to be in the cache. However, if a core on a socket tries to access a location which is on another socket's cache, it will use the interconnect between them to access it. If you have an unfortunate memory layout, this can make it so that you end up having a large percentage of the accesses using the interconnect (slower than cache access) and values get swapped between the caches constantly, which forces subsequent accesses to also use the interconnect. Another way to avoid this except using just one socket is for the designer of a program to consider NUMA nodes as separate processing units and design around that. Both should be processing separate data and they should only share small amounts of data for synchronization/communication. Then the caches will be much less affected.
- malkia 3y ago
  That's a pretty reasonable explanation, and one day I should sit down and write some artifical test/bench to get more details.
loeg 3y ago
Cache synchronization operations between CPU cores (MOESI) is cheaper (lower latency) between cores in the same socket than across sockets, often by a factor of 2x or more. But also limiting the program to one socket would reduce contention significantly. Both help.
Twirrim 3y ago
NUMA can be such a pain, especially because it's really difficult to account for in code, and all sorts of stuff you'd never think of can end up impacting your performance too, that usually you'd never even think of, e.g. the linux page cache. With the way that processors are going, with this focus on increased core counts etc, NUMA is increasingly being important to understand and account for, as processors are getting more "NUMA-ish" (to borrow a co-workers apt description). Especially Neoverse/Arm CPUs, etc.

by malkia 3y ago

BazookaMusic 3y ago

I might be wrong on this explanation, but the reason why it was faster might have been the following: During execution you had two kinds of memory locations, some in CPU caches and some in RAM. By running all the threads on one socket, everything accessed from the cache was just a fast cache access. Everything accessed from the memory was a slower memory load. Frequently loaded/stored locations will tend to go to the cache. In the NUMA setup, you would have a larger cache (more than one socket) which would mean that more locations were likely to be in the cache. However, if a core on a socket tries to access a location which is on another socket's cache, it will use the interconnect between them to access it. If you have an unfortunate memory layout, this can make it so that you end up having a large percentage of the accesses using the interconnect (slower than cache access) and values get swapped between the caches constantly, which forces subsequent accesses to also use the interconnect. Another way to avoid this except using just one socket is for the designer of a program to consider NUMA nodes as separate processing units and design around that. Both should be processing separate data and they should only share small amounts of data for synchronization/communication. Then the caches will be much less affected.

malkia 3y ago
That's a pretty reasonable explanation, and one day I should sit down and write some artifical test/bench to get more details.

loeg 3y ago

Cache synchronization operations between CPU cores (MOESI) is cheaper (lower latency) between cores in the same socket than across sockets, often by a factor of 2x or more. But also limiting the program to one socket would reduce contention significantly. Both help.

Twirrim 3y ago

NUMA can be such a pain, especially because it's really difficult to account for in code, and all sorts of stuff you'd never think of can end up impacting your performance too, that usually you'd never even think of, e.g. the linux page cache. With the way that processors are going, with this focus on increased core counts etc, NUMA is increasingly being important to understand and account for, as processors are getting more "NUMA-ish" (to borrow a co-workers apt description). Especially Neoverse/Arm CPUs, etc.