5 ms·

Linus is right. Don't use spinlocks in user-land. The scheduler can't know you're spinning on a lock, so it can't help you. Don't do dumb things -- spinning

Linus is right. Don't use spinlocks in user-land. The scheduler can't know you're spinning on a lock, so it can't help you. Don't do dumb things -- spinning on a lock in user-land is dumb. Mutexes are as fast as spinlocks in the uncontended case, so use mutexes.

shadowgovt 7y ago
Linus is right (for the kinds of code Linus tends to think about). There are good reasons videogame engines tend to use spinlocks, and if the takeaway is "Linux pessimizes for that use case," the implication is "Linux pessimizes for high-performance games atop its kernel."
- raverbashing 7y ago
  > "Linux pessimizes for high-performance games atop its kernel." Yes, because that's not the main focus of a general-purpose kernel. Which doesn't mean there aren't ways around it if your objective is running Linux for a specific application. The real time patches are in the mainline kernel since some time ago, but there are other options for softer RT at the kernel as well.
- quotemstr 7y ago
  No, there are not good reasons to use spinlocks in userland. Period. Even in a game. You always have the spinning while preempted problem. You can't blame a preempting operating system for preempting! Any situation you rig up where you don't preempt is very fragile. What you can do is make mutexes perfectly adaptive so that they switch to a blocking wait as soon as the lock owner itself gets preempted. I have a proposal to do that here: https://lore.kernel.org/lkml/CAKOZuesa338sc_=w6-wvro25idrSN_zx_zrHFyo-bDiORs7FZw@mail.gmail.com/ https://lore.kernel.org/lkml/CAKOZuesa338sc_=w6-wvro25idrSN_... You should always use waiting primitives when possible instead of just spinning. If you tell the kernel what you're doing, the kernel can help you. If you hide the wait DAG from the kernel, it can't.
  - shadowgovt 7y ago
    Unfortunately, texts on the topic of game engine development disagree with your assertion (https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=PA319#v=onepage&q&f=false https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=...). If people are learning bad practice from the texts, and the bad practice hits Linux's scheduler asymmetrically hard (relative to other game platforms), one can expect outsized impact on the performance of games atop Linux architectures.
    - rbanffy 7y ago
      > texts on the topic of game engine development disagree with your assertion We may need updated books.
    - cryptonector 7y ago
      Oh no! I guess the Linux kernel will need a fix then! Those texts can't be wrong!
      - shadowgovt 7y ago
        This is where technology meets social network effect. Which is easier (modifying the scheduler for better spinlock support or get developers to stop using spinlock when it's easy to implement and works in other OS targets) is an open question. "Don't use spinlock" is like "the user is holding it wrong" for API. The user can either choose to modify their behavior or throw up their hands and not port to Linux.
        quotemstr 7y ago
        You cannot "fix" the kernel. Fundamentally, the owner of spinlock can get preempted for a large number of different reasons, and in such a case, the spinlock will spin until the owning thread gets scheduled and runs again. The kernel can't prioritize re-scheduling the owning thread because it doesn't know that some other thread is waiting on that thread. Fundamentally, you need to tell the kernel what you're doing so it can help you. There's no fix here.
- the8472 7y ago
  No, linux pessimizes for that case if you do not cooperate with the scheduler. You can implement spinlocks in userspace under specific circumstances. You will need to mark your threads as realtime threads[0] and have a fallback to futex if the fast path (spinning) doesn't work out. And even then you need to benchmark on multiple machines and with realistic workloads (not microbenchmarks) to see whether it's actually worth it for your application. It's complex, linus goes into more detail here[1] [0] http://man7.org/linux/man-pages/man3/pthread_spin_init.3.html#NOTES http://man7.org/linux/man-pages/man3/pthread_spin_init.3.htm... [1] https://www.realworldtech.com/forum/?threadid=189711&curpostid=189755 https://www.realworldtech.com/forum/?threadid=189711&curpost...
  - shadowgovt 7y ago
    Do Windows, MacOS/iOS, and console OS's treat spinlocks more friendly than Linux?
    - CoolGuySteve 7y ago
      In Windows, CriticalSection can spin for a user-defined period of time, which is more or less what you want. Although it would probably make more sense for it to have a "spin until my quantum is over" option because how is the caller supposed to know that? https://docs.microsoft.com/en-us/cpp/cppcx/wrl/criticalsection-class?view=vs-2019#criticalsection https://docs.microsoft.com/en-us/cpp/cppcx/wrl/criticalsecti...
  - quotemstr 7y ago
    If you have a fallback to futex, then what you have isn't a spinlock, but an adaptive mutex (which spins, then blocks). Lots of operating systems have those already and you don't need to build your own.
    - shaklee3 7y ago
      Or just use a futex alone, since that mimics its behavior enough.
    - 7y ago
- falcolas 7y ago
  > There are good reasons videogame engines tend to use spinlocks Care to elaborate upon this statement? > "Linux pessimizes for high-performance games atop its kernel." Games can run at quite a high performance (which feels like a really odd statement) atop the general purpose Linux kernel. Even with the windows-specific APIs being emulated with Wine/Proton. There are many modern games, including Doom 2016 as I mentioned below) which run beautifully on Linux.
  - shadowgovt 7y ago
    > Care to elaborate upon this statement? I wouldn't redo the work of people who have literally written books on the topic (https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=PA319#v=onepage&q&f=false https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=...). Copyright 2019, so if it's not conventional wisdom that spinlocks are the right tool any longer, the "best practices" in game education are still teaching them, and OS's may need to account for the code actually written, not the code they wish were written.
    - danShumway 7y ago
      > OS's may need to account for the code actually written, not the code they wish were written. I don't think this should be asserted in such a black-and-white way. Yes, platforms need to accommodate the programming styles of users. But accommodating them blindly, when better solutions exist, is how you get bad APIs that age poorly. In platforms that are aiming for long-term stable APIs, it may make more sense in some cases to prioritize the cleanliness of the API over developer patterns, given that developer patterns are easier to change.
    - catalogia 7y ago
      If gamedev books are teaching specific quirks of the Windows scheduler, those books should be updated, rather than making linux emulate those quirks of Windows. Anticipated response: > "but that won't happen and gaming on linux will be too niche and will die." To be quite frank, that sounds like a satisfactory outcome to me. If Google wants a different status quo, they should throw more money at the problem rather than telling game developers to moan about it. You don't hear game developers moaning about this sort of shit when porting their games to the FreeBSD-backed PS4, presumably because Sony, unlike Google, actually gave a damn and made sure the platform worked and made sure developers knew how it works. If the requisite work has already been done on FreeBSD, then maybe Google should have decided to use that instead. Either way, it's on them. They made this mess for themselves.
- dfox 7y ago
  The reason why games tend to use spinlocks is that typical primary platforms for games do not have direct equivalent of Linux's default fast mutex.
deleted 7y ago
[deleted]

shadowgovt 7y ago

Linus is right (for the kinds of code Linus tends to think about). There are good reasons videogame engines tend to use spinlocks, and if the takeaway is "Linux pessimizes for that use case," the implication is "Linux pessimizes for high-performance games atop its kernel."

raverbashing 7y ago
> "Linux pessimizes for high-performance games atop its kernel." Yes, because that's not the main focus of a general-purpose kernel. Which doesn't mean there aren't ways around it if your objective is running Linux for a specific application. The real time patches are in the mainline kernel since some time ago, but there are other options for softer RT at the kernel as well.
quotemstr 7y ago
No, there are not good reasons to use spinlocks in userland. Period. Even in a game. You always have the spinning while preempted problem. You can't blame a preempting operating system for preempting! Any situation you rig up where you don't preempt is very fragile. What you can do is make mutexes perfectly adaptive so that they switch to a blocking wait as soon as the lock owner itself gets preempted. I have a proposal to do that here: https://lore.kernel.org/lkml/CAKOZuesa338sc_=w6-wvro25idrSN_zx_zrHFyo-bDiORs7FZw@mail.gmail.com/ https://lore.kernel.org/lkml/CAKOZuesa338sc_=w6-wvro25idrSN_... You should always use waiting primitives when possible instead of just spinning. If you tell the kernel what you're doing, the kernel can help you. If you hide the wait DAG from the kernel, it can't.
- shadowgovt 7y ago
  Unfortunately, texts on the topic of game engine development disagree with your assertion (https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=PA319#v=onepage&q&f=false https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=...). If people are learning bad practice from the texts, and the bad practice hits Linux's scheduler asymmetrically hard (relative to other game platforms), one can expect outsized impact on the performance of games atop Linux architectures.
  - rbanffy 7y ago
    > texts on the topic of game engine development disagree with your assertion We may need updated books.
  - cryptonector 7y ago
    Oh no! I guess the Linux kernel will need a fix then! Those texts can't be wrong!
    - shadowgovt 7y ago
      This is where technology meets social network effect. Which is easier (modifying the scheduler for better spinlock support or get developers to stop using spinlock when it's easy to implement and works in other OS targets) is an open question. "Don't use spinlock" is like "the user is holding it wrong" for API. The user can either choose to modify their behavior or throw up their hands and not port to Linux.
      - quotemstr 7y ago
        You cannot "fix" the kernel. Fundamentally, the owner of spinlock can get preempted for a large number of different reasons, and in such a case, the spinlock will spin until the owning thread gets scheduled and runs again. The kernel can't prioritize re-scheduling the owning thread because it doesn't know that some other thread is waiting on that thread. Fundamentally, you need to tell the kernel what you're doing so it can help you. There's no fix here.
the8472 7y ago
No, linux pessimizes for that case if you do not cooperate with the scheduler. You can implement spinlocks in userspace under specific circumstances. You will need to mark your threads as realtime threads[0] and have a fallback to futex if the fast path (spinning) doesn't work out. And even then you need to benchmark on multiple machines and with realistic workloads (not microbenchmarks) to see whether it's actually worth it for your application. It's complex, linus goes into more detail here[1] [0] http://man7.org/linux/man-pages/man3/pthread_spin_init.3.html#NOTES http://man7.org/linux/man-pages/man3/pthread_spin_init.3.htm... [1] https://www.realworldtech.com/forum/?threadid=189711&curpostid=189755 https://www.realworldtech.com/forum/?threadid=189711&curpost...
- shadowgovt 7y ago
  Do Windows, MacOS/iOS, and console OS's treat spinlocks more friendly than Linux?
  - CoolGuySteve 7y ago
    In Windows, CriticalSection can spin for a user-defined period of time, which is more or less what you want. Although it would probably make more sense for it to have a "spin until my quantum is over" option because how is the caller supposed to know that? https://docs.microsoft.com/en-us/cpp/cppcx/wrl/criticalsection-class?view=vs-2019#criticalsection https://docs.microsoft.com/en-us/cpp/cppcx/wrl/criticalsecti...
- quotemstr 7y ago
  If you have a fallback to futex, then what you have isn't a spinlock, but an adaptive mutex (which spins, then blocks). Lots of operating systems have those already and you don't need to build your own.
  - shaklee3 7y ago
    Or just use a futex alone, since that mimics its behavior enough.
  - 7y ago
falcolas 7y ago
> There are good reasons videogame engines tend to use spinlocks Care to elaborate upon this statement? > "Linux pessimizes for high-performance games atop its kernel." Games can run at quite a high performance (which feels like a really odd statement) atop the general purpose Linux kernel. Even with the windows-specific APIs being emulated with Wine/Proton. There are many modern games, including Doom 2016 as I mentioned below) which run beautifully on Linux.
- shadowgovt 7y ago
  > Care to elaborate upon this statement? I wouldn't redo the work of people who have literally written books on the topic (https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=PA319#v=onepage&q&f=false https://books.google.com/books?id=1g1mDwAAQBAJ&pg=PA319&lpg=...). Copyright 2019, so if it's not conventional wisdom that spinlocks are the right tool any longer, the "best practices" in game education are still teaching them, and OS's may need to account for the code actually written, not the code they wish were written.
  - danShumway 7y ago
    > OS's may need to account for the code actually written, not the code they wish were written. I don't think this should be asserted in such a black-and-white way. Yes, platforms need to accommodate the programming styles of users. But accommodating them blindly, when better solutions exist, is how you get bad APIs that age poorly. In platforms that are aiming for long-term stable APIs, it may make more sense in some cases to prioritize the cleanliness of the API over developer patterns, given that developer patterns are easier to change.
  - catalogia 7y ago
    If gamedev books are teaching specific quirks of the Windows scheduler, those books should be updated, rather than making linux emulate those quirks of Windows. Anticipated response: > "but that won't happen and gaming on linux will be too niche and will die." To be quite frank, that sounds like a satisfactory outcome to me. If Google wants a different status quo, they should throw more money at the problem rather than telling game developers to moan about it. You don't hear game developers moaning about this sort of shit when porting their games to the FreeBSD-backed PS4, presumably because Sony, unlike Google, actually gave a damn and made sure the platform worked and made sure developers knew how it works. If the requisite work has already been done on FreeBSD, then maybe Google should have decided to use that instead. Either way, it's on them. They made this mess for themselves.
dfox 7y ago
The reason why games tend to use spinlocks is that typical primary platforms for games do not have direct equivalent of Linux's default fast mutex.

deleted 7y ago

[deleted]