5 ms·

Atomic polling intervals for highly concurrent workloads

31 points by mmastrac 2y ago

forrestthewoods 2y ago
If you're mutating the same atomic across 100 threads you're gonna have a bad time. Atomic doesn't mean fast. Hell, that might even be slower than using a mutex? Despite what you'd hope, atomic doesn't mean fast.
- jeffbee 2y ago
  Indeed, the article is quite misleading by supposing that atomic variables are magic at the hardware level. On some hardware yes, on others no. As an example the original AWS Graviton is vanilla armv8-a without atomics, so atomic concepts in languages are simulated with retry loops having unbounded latency. Your thread that wants to increment an atomic may be blocked literally forever if there are enough contenders for that cache line. Library mutex implementations are usually hybrids with limited atomic spinlocks, with pauses or sleeps, then fallbacks to heavyweight synchronization with fairness features such as futex. On certain Intel hardware we also have lightweight userspace notification when a given address has been written. That can be used for adaptive wakeup on atomic values but it is "lightweight" relative to futex, since it still requires a rdtsc and still amounts to an efficient spinloop. On Intel Atom CPUs you can do this with up to 4 threads for free, but 100 threads will still be a mess.
- jay-barronville 2y ago
  > Hell, that might even be slower than using a mutex? It depends, of course, on how the mutex is implemented, but yes, this is the reality a lot of times. The thing about atomics is that they can appear simple to naïve eyes, but the semantics are very difficult to get right (especially if you want your code to be portable across architectures).
- helpfulContrib 2y ago
  [dead]
juancn 2y ago
Atomics are not magical, usually there's a type of "lock" at the cache line level (actually a lot of message passing across cores). Using the memory ordering relaxed basically only guarantees there won't be word tearing. For example consider the following: // T1: r1 = load_relaxed(y); // A store_relaxed(x, r1); // B // T2: r2 = load_relaxed(x); // C store_relaxed(y, 42); // D Which implies that it's legal to read r1 == r2 == 42, because even if: - in T1 A happens before B (A ~> B) - in T2 C happens before D (C ~> D) nothing prevents D from happening before A in the modification order of y and C happening before B on the modification order of x. In any case, on Intel processors, most accesses not crossing cache line boundaries are somewhat atomic beyond relaxed, which brings about another phenomena: false sharing. Which essentially boils down to a performance penalty due to two threads accessing data on the same cache line, even if it's not the same word. Another thing that can affect measurements like this is task/core affinity. The cost to coordinate cache line synchronization is different if both are on the same core but different hyper-threads, two cores on the same socket (with some variation on which cores) and if they are on different sockets (big/little architectures bring their own complications too). In the article's example, it would be valid for the counter to never be flushed (not on x86), as long as there's enough cache capacity, effectively making the polling thread read always the same number. Although this is very unlikely on realistic workloads. I would have used release/acquire semantics to get predictable capture of data and a not too aggressive polling behavior so as not to compete for CPU caches with useful workloads.
yencabulator 2y ago
Sounds like the author hasn't yet understood one level below. CPUs lock cachelines with MESI, and forcing a cacheline exclusive to you every 10ms will obviously hurt performance. https://en.wikipedia.org/wiki/MESI_protocol https://en.wikipedia.org/wiki/MESI_protocol

31 points by mmastrac 2y ago

If you're mutating the same atomic across 100 threads you're gonna have a bad time. Atomic doesn't mean fast. Hell, that might even be slower than using a mutex? Despite what you'd hope, atomic doesn't mean fast.

jeffbee 2y ago
Indeed, the article is quite misleading by supposing that atomic variables are magic at the hardware level. On some hardware yes, on others no. As an example the original AWS Graviton is vanilla armv8-a without atomics, so atomic concepts in languages are simulated with retry loops having unbounded latency. Your thread that wants to increment an atomic may be blocked literally forever if there are enough contenders for that cache line. Library mutex implementations are usually hybrids with limited atomic spinlocks, with pauses or sleeps, then fallbacks to heavyweight synchronization with fairness features such as futex. On certain Intel hardware we also have lightweight userspace notification when a given address has been written. That can be used for adaptive wakeup on atomic values but it is "lightweight" relative to futex, since it still requires a rdtsc and still amounts to an efficient spinloop. On Intel Atom CPUs you can do this with up to 4 threads for free, but 100 threads will still be a mess.
jay-barronville 2y ago
> Hell, that might even be slower than using a mutex? It depends, of course, on how the mutex is implemented, but yes, this is the reality a lot of times. The thing about atomics is that they can appear simple to naïve eyes, but the semantics are very difficult to get right (especially if you want your code to be portable across architectures).
helpfulContrib 2y ago
[dead]

juancn 2y ago

Atomics are not magical, usually there's a type of "lock" at the cache line level (actually a lot of message passing across cores). Using the memory ordering relaxed basically only guarantees there won't be word tearing. For example consider the following: // T1: r1 = load_relaxed(y); // A store_relaxed(x, r1); // B // T2: r2 = load_relaxed(x); // C store_relaxed(y, 42); // D Which implies that it's legal to read r1 == r2 == 42, because even if: - in T1 A happens before B (A ~> B) - in T2 C happens before D (C ~> D) nothing prevents D from happening before A in the modification order of y and C happening before B on the modification order of x. In any case, on Intel processors, most accesses not crossing cache line boundaries are somewhat atomic beyond relaxed, which brings about another phenomena: false sharing. Which essentially boils down to a performance penalty due to two threads accessing data on the same cache line, even if it's not the same word. Another thing that can affect measurements like this is task/core affinity. The cost to coordinate cache line synchronization is different if both are on the same core but different hyper-threads, two cores on the same socket (with some variation on which cores) and if they are on different sockets (big/little architectures bring their own complications too). In the article's example, it would be valid for the counter to never be flushed (not on x86), as long as there's enough cache capacity, effectively making the polling thread read always the same number. Although this is very unlikely on realistic workloads. I would have used release/acquire semantics to get predictable capture of data and a not too aggressive polling behavior so as not to compete for CPU caches with useful workloads.

yencabulator 2y ago

Sounds like the author hasn't yet understood one level below. CPUs lock cachelines with MESI, and forcing a cacheline exclusive to you every 10ms will obviously hurt performance. https://en.wikipedia.org/wiki/MESI_protocol https://en.wikipedia.org/wiki/MESI_protocol