5 ms·

They're talking past each other. Malte is really saying that yield() does what he wants on Windows and BSD, and Linus is giving a bunch of excuses for why Linux

by gok 7y ago

They're talking past each other. Malte is really saying that yield() does what he wants on Windows and BSD, and Linus is giving a bunch of excuses for why Linux doesn't work that way.

muststopmyths 7y ago
EXACTLY! I wish someone would just explain why the behavior is different. Does linux only sched_yield on the same NUMA node ? (Linus' reply seemed to imply that). Windows will not pin threads unless you set processor affinity, AFAIK, so maybe that's the difference. There are runnable threads that are not being run. It would be nice to know why. Tell a person their code is wrong and you may have saved them from one mistake. Tell them why their code is wrong and you will save them from a dozen.
- wyldfire 7y ago
  > Does linux only sched_yield on the same NUMA node ? (Linus' reply seemed to imply that). Perhaps Linus' concern is that sched_yield doesn't guarantee any particular behavior, it just happens to be implemented that way now.
- Dylan16807 7y ago
  > Does linux only sched_yield on the same NUMA node ? > There are runnable threads that are not being run. It would be nice to know why. More extreme. It only applies to the same core, more or less. The original benchmarks are on a one-node CPU, after all.
pizlonator 7y ago
I’m pretty sure that the Linux behavior makes sched_yield more suitable to the 40-yields optimization I use for locks. I don’t buy claims about lock behavior based on microbenchmarks. Microbenchmarks have wildly different behavior from one another so it stands to reason that they have wildly different behavior from real code. Linus’s excuse is that if you use a real lock then shit is good. That’s a pretty good excuse IMO.
- gok 7y ago
  Yeah Linus is correct here, he's just not directly addressing why it works on some other schedulers.
  - pizlonator 7y ago
    Agreed.
  - spamizbad 7y ago
    One reason they might act that way: They have nowhere near the amount of NUMA or cache locality optimization work the Linux kernel possesses. Thinking of the 2nd-generation Threadripper issues in Windows in particular.
    - pizlonator 7y ago
      That's quite likely. I think that the sched_yield behavior is throughput-optimized, which is consistent with what you get when an OS is server-optimized like Linux.
cycloptic 7y ago
There's no excuses, the behavior of the scheduler is documented and straightforward. Up thread a twitter user [0] noted that you can somewhat replicate the behavior of windows by replacing sched_yield() with sleep(0). The original benchmark skips this detail and just assumes the C++ wrapper will exhibit the same behavior on all OSes with all schedulers, which is clearly wrong. [0]: https://nitter.net/dotstdy/status/1213582199615901696#m https://nitter.net/dotstdy/status/1213582199615901696#m
soraminazuki 7y ago
The point is Windows and BSD isn't doing "what he wants," because the benchmark in question isn't producing any meaningful data for two reasons. First, it's not even measuring what Malte said he was measuring. Second, the benchmark assumes a single threaded HW model, and optimizing for it would very likely slow down performance for modern real-world workloads.
- gok 7y ago
  Ignore the benchmark for a moment; the original issue is that a real program (the game Rage 2) uses spin locks, and experiences really terrible hangs on Linux and not on the other kernels. What he wanted was for that to not happen. The upshot is that there really is no way to get spinlocks to not hang on Linux. Now, there's a very good argument that spinlocks are just dumb, and therefore that real game is broken.

by gok 7y ago

They're talking past each other. Malte is really saying that yield() does what he wants on Windows and BSD, and Linus is giving a bunch of excuses for why Linux doesn't work that way.

muststopmyths 7y ago

EXACTLY! I wish someone would just explain why the behavior is different. Does linux only sched_yield on the same NUMA node ? (Linus' reply seemed to imply that). Windows will not pin threads unless you set processor affinity, AFAIK, so maybe that's the difference. There are runnable threads that are not being run. It would be nice to know why. Tell a person their code is wrong and you may have saved them from one mistake. Tell them why their code is wrong and you will save them from a dozen.

wyldfire 7y ago
> Does linux only sched_yield on the same NUMA node ? (Linus' reply seemed to imply that). Perhaps Linus' concern is that sched_yield doesn't guarantee any particular behavior, it just happens to be implemented that way now.
Dylan16807 7y ago
> Does linux only sched_yield on the same NUMA node ? > There are runnable threads that are not being run. It would be nice to know why. More extreme. It only applies to the same core, more or less. The original benchmarks are on a one-node CPU, after all.

pizlonator 7y ago

I’m pretty sure that the Linux behavior makes sched_yield more suitable to the 40-yields optimization I use for locks. I don’t buy claims about lock behavior based on microbenchmarks. Microbenchmarks have wildly different behavior from one another so it stands to reason that they have wildly different behavior from real code. Linus’s excuse is that if you use a real lock then shit is good. That’s a pretty good excuse IMO.

gok 7y ago
Yeah Linus is correct here, he's just not directly addressing why it works on some other schedulers.
- pizlonator 7y ago
  Agreed.
- spamizbad 7y ago
  One reason they might act that way: They have nowhere near the amount of NUMA or cache locality optimization work the Linux kernel possesses. Thinking of the 2nd-generation Threadripper issues in Windows in particular.
  - pizlonator 7y ago
    That's quite likely. I think that the sched_yield behavior is throughput-optimized, which is consistent with what you get when an OS is server-optimized like Linux.

cycloptic 7y ago

There's no excuses, the behavior of the scheduler is documented and straightforward. Up thread a twitter user [0] noted that you can somewhat replicate the behavior of windows by replacing sched_yield() with sleep(0). The original benchmark skips this detail and just assumes the C++ wrapper will exhibit the same behavior on all OSes with all schedulers, which is clearly wrong. [0]: https://nitter.net/dotstdy/status/1213582199615901696#m https://nitter.net/dotstdy/status/1213582199615901696#m

soraminazuki 7y ago

The point is Windows and BSD isn't doing "what he wants," because the benchmark in question isn't producing any meaningful data for two reasons. First, it's not even measuring what Malte said he was measuring. Second, the benchmark assumes a single threaded HW model, and optimizing for it would very likely slow down performance for modern real-world workloads.

gok 7y ago
Ignore the benchmark for a moment; the original issue is that a real program (the game Rage 2) uses spin locks, and experiences really terrible hangs on Linux and not on the other kernels. What he wanted was for that to not happen. The upshot is that there really is no way to get spinlocks to not hang on Linux. Now, there's a very good argument that spinlocks are just dumb, and therefore that real game is broken.