5 ms·

>Critical section under 100ns, low contention (2-4 threads): Spinlock. You’ll waste less time spinning than you would on a context switch. If your sections are

>Critical section under 100ns, low contention (2-4 threads): Spinlock. You’ll waste less time spinning than you would on a context switch. If your sections are that short then you can use a hybrid mutex and never actually park. Unless you're wrong about how long things take, in which case you'll save yourself. >alignas(64) in C++ std::hardware_destructive_interference_size Exists so you don't have to guess, although in practice it'll basically always be 64. The code samples also don't obey the basic best practices for spinlocks for x86_64 or arm64. Spinlocks should perform a relaxed read in the loop, and only attempt a compare and set with acquire order if the first check shows the lock is unowned. This avoids hammering the CPU with cache coherency traffic. Similarly the x86 PAUSE instruction isn't mentioned, even though it exist specifically to signal spin sections to the CPU. Spinlocks outside the kernel are a bad idea in almost all cases, except dedicated nonpreemptable cases; use a hybrid mutex. Spinning for consumer threads can be done in specialty exclusive thread per core cases where you want to minimize wakeup costs, but that's not the same as a spinlock which would cause any contending thread to spin.

magicalhippo 10mo ago
> Spinlocks outside the kernel are a bad idea in almost all cases, except dedicated nonpreemptable cases; use a hybrid mutex Yeah, pure spinlocks in user-space programs is a big no-no in my book. If you're on the happy path then it costs you nothing extra in terms of performance, and if you for some reason slide off the happy path you have a sensible fall-back.
raggi 10mo ago
> Spinlocks outside the kernel are a bad idea in almost all cases, except dedicated nonpreemptable cases; use a hybrid mutex. Spinning for consumer threads can be done in specialty exclusive thread per core cases where you want to minimize wakeup costs, but that's not the same as a spinlock which would cause any contending thread to spin. Very much this. Spins benchmark well but scale poorly.
charleshn 10mo ago
> std::hardware_destructive_interference_size Exists so you don't have to guess, although in practice it'll basically always be 64. Unfortunately it's not quite true, do to e.g. spacial prefetching [0]. See e.g. Folly's definition [1]. [0] https://community.intel.com/t5/Intel-Moderncode-for-Parallel/Hardware-prefetch-and-shared-multi-core-resources-on-Xeon/td-p/1074003 https://community.intel.com/t5/Intel-Moderncode-for-Parallel... [1] https://github.com/facebook/folly/blob/d2e6fe65dfd6b30a9d504d0409ac733cbaa73125/folly/lang/Align.h#L172 https://github.com/facebook/folly/blob/d2e6fe65dfd6b30a9d504...
saagarjha 10mo ago
> std::hardware_destructive_interference_size Of course, this is just the number the compiler thinks is good. It’s not necessarily the number that is actually good for your target machine.
surajrmal 10mo ago
Hybrid locks are also bad for overall system performance by maximizing local application performance. There is a reason default lock implementations from OS don't spin even a little bit.
- menaerus 10mo ago
  > There is a reason default lock implementations from OS don't spin even a little bit. glibc pthread mutex uses a user-space spinlock to mitigate the syscall cost for uncontended cases.
- charleslmunger 10mo ago
  That depends on your workload. If you're making a game that's expected to use near 100% of system resources, or a real time service pinned to specific cores, your local application is the overall system.
  - surajrmal 10mo ago
    Totally agree. However it's important to differentiate those workloads from the average workload which is to participate in a larger system.
- nly 10mo ago
  GNU libc posix mutexes do spin...
  - surajrmal 10mo ago
    And I think it'd a poor choice that causes worse system performance. Android's bionic doesn't spin, nor does Windows or Fuchsia. Avoiding the syscall overhead is generally detrimental to overall system performance especially when the CPU load is high.
- imtringued 10mo ago
  This is nonsense. If the lock hasn't been acquired, you don't spin to begin with and if the lock has been acquired and the lock is being released shortly after, the spinning avoids a context switch. If the maximum number of retries has been reached, the thread was going to sleep anyway and starts scheduling the next thread (which was only delayed by the few attempted spins). This means in the worst case the next spin will only happen once all the other queued up threads have had their turn and that's assuming you're immediately running into another acquired lock.
nly 10mo ago
The PAUSE instruction isn't actually as good as it used to be. In, iirc, Skylake Intel massively increased the latency to improve utilisation under hyperthreading. The latency of this instruction is now really high. Most people using spinlocks really care about latency, and many will have hyperthreading disabled to reduce jitter
- SkiFire13 10mo ago
  If the PAUSE instruction is too fast doesn't that kinda defeat its purpose?
  - menaerus 10mo ago
    Yeah, I think so too now that I read some documentation about it. It appears that the main issue with the spinlock pattern is that it inhibits "a severe performance penalty when exiting the [spinlock] loop because it [CPU] detects a possible memory order violation." [0]. ~10 years ago, on Haswell, it took ~9 cycles to retire, and from Skylake onward, with some exceptions, it takes a magnitude more - ~140 cycles. These numbers alone suggests that it really messes up hard with the CPU pipeline, perhaps BP (?) or speculative execution (?) or both (?) such that it will basically force the CPU to flush the whole pipeline. This is at least how I read this. I will remember this instruction as "damage control" instruction from now on. [0] https://www.felixcloutier.com/x86/pause https://www.felixcloutier.com/x86/pause
  - nly 10mo ago
    Not sure if you'll see this now, but the actual reason you want to use it is as a speculation barrier and a hint to various predictors. Lfence is the better choice these days.
menaerus 10mo ago
Some things from the article are debatable for sure, and some are maybe missing like the one you mention with PAUSE instruction, which I also have not been aware of, but generally speaking I thought it was a really good content. Lean system engineering skills applied to real world problems. I especially appreciated the examples of large-scale infra codebases doing it in practice.

magicalhippo 10mo ago

> Spinlocks outside the kernel are a bad idea in almost all cases, except dedicated nonpreemptable cases; use a hybrid mutex Yeah, pure spinlocks in user-space programs is a big no-no in my book. If you're on the happy path then it costs you nothing extra in terms of performance, and if you for some reason slide off the happy path you have a sensible fall-back.

raggi 10mo ago

> Spinlocks outside the kernel are a bad idea in almost all cases, except dedicated nonpreemptable cases; use a hybrid mutex. Spinning for consumer threads can be done in specialty exclusive thread per core cases where you want to minimize wakeup costs, but that's not the same as a spinlock which would cause any contending thread to spin. Very much this. Spins benchmark well but scale poorly.

charleshn 10mo ago

> std::hardware_destructive_interference_size Exists so you don't have to guess, although in practice it'll basically always be 64. Unfortunately it's not quite true, do to e.g. spacial prefetching [0]. See e.g. Folly's definition [1]. [0] https://community.intel.com/t5/Intel-Moderncode-for-Parallel/Hardware-prefetch-and-shared-multi-core-resources-on-Xeon/td-p/1074003 https://community.intel.com/t5/Intel-Moderncode-for-Parallel... [1] https://github.com/facebook/folly/blob/d2e6fe65dfd6b30a9d504d0409ac733cbaa73125/folly/lang/Align.h#L172 https://github.com/facebook/folly/blob/d2e6fe65dfd6b30a9d504...

saagarjha 10mo ago

> std::hardware_destructive_interference_size Of course, this is just the number the compiler thinks is good. It’s not necessarily the number that is actually good for your target machine.

surajrmal 10mo ago

Hybrid locks are also bad for overall system performance by maximizing local application performance. There is a reason default lock implementations from OS don't spin even a little bit.

menaerus 10mo ago
> There is a reason default lock implementations from OS don't spin even a little bit. glibc pthread mutex uses a user-space spinlock to mitigate the syscall cost for uncontended cases.
charleslmunger 10mo ago
That depends on your workload. If you're making a game that's expected to use near 100% of system resources, or a real time service pinned to specific cores, your local application is the overall system.
- surajrmal 10mo ago
  Totally agree. However it's important to differentiate those workloads from the average workload which is to participate in a larger system.
nly 10mo ago
GNU libc posix mutexes do spin...
- surajrmal 10mo ago
  And I think it'd a poor choice that causes worse system performance. Android's bionic doesn't spin, nor does Windows or Fuchsia. Avoiding the syscall overhead is generally detrimental to overall system performance especially when the CPU load is high.
imtringued 10mo ago
This is nonsense. If the lock hasn't been acquired, you don't spin to begin with and if the lock has been acquired and the lock is being released shortly after, the spinning avoids a context switch. If the maximum number of retries has been reached, the thread was going to sleep anyway and starts scheduling the next thread (which was only delayed by the few attempted spins). This means in the worst case the next spin will only happen once all the other queued up threads have had their turn and that's assuming you're immediately running into another acquired lock.

nly 10mo ago

The PAUSE instruction isn't actually as good as it used to be. In, iirc, Skylake Intel massively increased the latency to improve utilisation under hyperthreading. The latency of this instruction is now really high. Most people using spinlocks really care about latency, and many will have hyperthreading disabled to reduce jitter

SkiFire13 10mo ago
If the PAUSE instruction is too fast doesn't that kinda defeat its purpose?
- menaerus 10mo ago
  Yeah, I think so too now that I read some documentation about it. It appears that the main issue with the spinlock pattern is that it inhibits "a severe performance penalty when exiting the [spinlock] loop because it [CPU] detects a possible memory order violation." [0]. ~10 years ago, on Haswell, it took ~9 cycles to retire, and from Skylake onward, with some exceptions, it takes a magnitude more - ~140 cycles. These numbers alone suggests that it really messes up hard with the CPU pipeline, perhaps BP (?) or speculative execution (?) or both (?) such that it will basically force the CPU to flush the whole pipeline. This is at least how I read this. I will remember this instruction as "damage control" instruction from now on. [0] https://www.felixcloutier.com/x86/pause https://www.felixcloutier.com/x86/pause
- nly 10mo ago
  Not sure if you'll see this now, but the actual reason you want to use it is as a speculation barrier and a hint to various predictors. Lfence is the better choice these days.

menaerus 10mo ago

Some things from the article are debatable for sure, and some are maybe missing like the one you mention with PAUSE instruction, which I also have not been aware of, but generally speaking I thought it was a really good content. Lean system engineering skills applied to real world problems. I especially appreciated the examples of large-scale infra codebases doing it in practice.