3 ms·

The whole point of spinning is to avoid the expense of a context switch by putting the thread to sleep, letting the OS schedule another. If you're going to spin

by eloff 8y ago

The whole point of spinning is to avoid the expense of a context switch by putting the thread to sleep, letting the OS schedule another. If you're going to spin for 10ms, that's longer than the timeslice most threads run for. That is going to be well above the overhead of any context switch. It makes no sense.

dragontamer 8y ago
> That is going to be well above the overhead of any context switch. Is it though? The "pause" instruction gives all resources to the hyperthread sibling. So under high-utilization (ie: your core has a hyperthread-sibling ready and willing), a "pause" instruction is basically a super-low latency task-switch. I wouldn't be surprised that a "spinlock with pause" would be better, even in 5ms to 10ms chunks of time (Windows has 15ms quantas, and may task switch as late as 30ms to 75ms depending on the situation). Because its better to do the hardware-assisted "pause" task switch rather than actually cause the OS-scheduler to get involved. In fact, that's what Microsoft AND Intel have concluded!! To get better real-world metrics, both Microsoft AND Intel felt like the "pause" instruction should wait longer! Skylake-X increases the latency, while .NET created this exponential-backoff. I bet it was good in isolation, but when combined, the overall effect was too much pausing.
- eloff 8y ago
  Giving resources to the sibling hyperthread via pause is not the kind of context switch I'm talking about. I'm talking about the expensive context switch when the OS scheduler must swap out a running thread for another. That carries a steep obvious penalty switching between user space and the kernel and back, but actually can involve much heavier penalties where the cache and TLB are concerned - depending what thread runs next. Pausing for as much as 10ms could be better under some circumstances, but that would seem to me to be heavier than a context-switch. Whether the system actually achieves better throughput or not in that case would be workload dependent. Worst case the hyperthread is also running the spin lock loop and no useful work at all is accomplished on the core.
  - dragontamer 8y ago
    > Giving resources to the sibling hyperthread via pause is not the kind of context switch I'm talking about. Isn't this entire thread about "pause" ?? Anyway, i think we both understand the difference between "context switch" and "pause" here. > Pausing for as much as 10ms could be better under some circumstances, but that would seem to me to be heavier than a context-switch. Whether the system actually achieves better throughput or not in that case would be workload dependent. Seems reasonable. Ideally, threads / cores shouldn't be blocked that long, so 10ms+ blocked periods is definitely an anomaly. But whether or not a context switch is actually worth it? Ehhh, its hard to say for sure.
- kllrnohj 8y ago
  > Is it though? Yes, definitely, beyond any doubt. The .NET approach only makes sense if you have exactly the same number of active threads as CPU "cores" and if your CPU always supports hyperthreading. If either of those is not true then what .NET is doing is a severe CPU bottleneck when the lock is actually contended. And it is incredibly unlikely that that exact situation happens in the wild. You almost always have more than <cpu core count> threads/processes active in heavy multithreading workloads, and you definitely can't assume everyone has hyperthreading as that's a feature commonly cut from mid & low range CPUs. For context here a thread context switch via something like futex is in the sub-10us range. So a 5ms spinlock robs you of ~4.9ms of usable CPU throughput that a different thread/process could have been doing instead. The timeout levels here are just shy of an eternity. 5ms is a really, really long amount of time. By way of comparison pthread_mutex_lock in glibc will do one pause per attempt for at most 100 attempts. So under skylake it has a max pause duration of around 3-5us, depending on clock speed, which is pretty reasonable. It roughly aligns with the cost of a context switch, so once it starts spinning for longer than a context switch it switches to a context switch instead.

by eloff 8y ago

dragontamer 8y ago

> That is going to be well above the overhead of any context switch. Is it though? The "pause" instruction gives all resources to the hyperthread sibling. So under high-utilization (ie: your core has a hyperthread-sibling ready and willing), a "pause" instruction is basically a super-low latency task-switch. I wouldn't be surprised that a "spinlock with pause" would be better, even in 5ms to 10ms chunks of time (Windows has 15ms quantas, and may task switch as late as 30ms to 75ms depending on the situation). Because its better to do the hardware-assisted "pause" task switch rather than actually cause the OS-scheduler to get involved. In fact, that's what Microsoft AND Intel have concluded!! To get better real-world metrics, both Microsoft AND Intel felt like the "pause" instruction should wait longer! Skylake-X increases the latency, while .NET created this exponential-backoff. I bet it was good in isolation, but when combined, the overall effect was too much pausing.

eloff 8y ago
Giving resources to the sibling hyperthread via pause is not the kind of context switch I'm talking about. I'm talking about the expensive context switch when the OS scheduler must swap out a running thread for another. That carries a steep obvious penalty switching between user space and the kernel and back, but actually can involve much heavier penalties where the cache and TLB are concerned - depending what thread runs next. Pausing for as much as 10ms could be better under some circumstances, but that would seem to me to be heavier than a context-switch. Whether the system actually achieves better throughput or not in that case would be workload dependent. Worst case the hyperthread is also running the spin lock loop and no useful work at all is accomplished on the core.
- dragontamer 8y ago
  > Giving resources to the sibling hyperthread via pause is not the kind of context switch I'm talking about. Isn't this entire thread about "pause" ?? Anyway, i think we both understand the difference between "context switch" and "pause" here. > Pausing for as much as 10ms could be better under some circumstances, but that would seem to me to be heavier than a context-switch. Whether the system actually achieves better throughput or not in that case would be workload dependent. Seems reasonable. Ideally, threads / cores shouldn't be blocked that long, so 10ms+ blocked periods is definitely an anomaly. But whether or not a context switch is actually worth it? Ehhh, its hard to say for sure.
kllrnohj 8y ago
> Is it though? Yes, definitely, beyond any doubt. The .NET approach only makes sense if you have exactly the same number of active threads as CPU "cores" and if your CPU always supports hyperthreading. If either of those is not true then what .NET is doing is a severe CPU bottleneck when the lock is actually contended. And it is incredibly unlikely that that exact situation happens in the wild. You almost always have more than <cpu core count> threads/processes active in heavy multithreading workloads, and you definitely can't assume everyone has hyperthreading as that's a feature commonly cut from mid & low range CPUs. For context here a thread context switch via something like futex is in the sub-10us range. So a 5ms spinlock robs you of ~4.9ms of usable CPU throughput that a different thread/process could have been doing instead. The timeout levels here are just shy of an eternity. 5ms is a really, really long amount of time. By way of comparison pthread_mutex_lock in glibc will do one pause per attempt for at most 100 attempts. So under skylake it has a max pause duration of around 3-5us, depending on clock speed, which is pretty reasonable. It roughly aligns with the cost of a context switch, so once it starts spinning for longer than a context switch it switches to a context switch instead.