3 ms·
This blog post has some good discussion about what makes a good locking primitive. On macOS, os_unfair_lock is generally a good choice for many applications. Th
by saagarjha 4y ago
This blog post has some good discussion about what makes a good locking primitive. On macOS, os_unfair_lock is generally a good choice for many applications. That said,
> At this point, you might wonder if os_unfair_lock – possibly coupled with the undocumented flags – would be a good fit for your codebase. My answer is likely yes but you’ll have to be careful when using it.
I would hesitate to give this advice. The overwhelming majority of applications on macOS do not have performance profiles like Firefox does. The system allocator (rather than jemalloc) is the right choice for all but very few use cases. There are a vanishingly small number of usecases where using the undocumented flags is appropriate.
The rules are different when you’re a popular browser with an existing relationship with Apple and dozens of smart engineers who can (at least in theory) understand the consequences of using this API, and leverage an existing relationship with Apple to ensure that it doesn’t become a liability in the future as the platform evolves. For pretty much everyone else, this should be nothing more than a “wow, neat” blog post.
- lapcat 4y ago> leverage an existing relationship with Apple to ensure that it doesn’t become a liability in the future as the platform evolves I sincerely doubt that Firefox can leverage anything with Apple, or that Firefox engineers have much of a relationship at all with Apple. Who does, really. I'm not sure it matters though. Many smaller Mac developers have a long history of using private API, and it's largely fine. There's no truly "safe path" with Apple, because Apple does whatever it wants whenever it wants, and I've seen them break "supported" API plenty of times, and leave it in a broken state. But the risks of any given brokenness are small, and you can't predict what will break from year to year, so what can you do except go with what works, and hope for the best. It's certainly fair to say that the majority of applications on macOS do not have the performance profiles of Firefox, but it's also the case that the majority of applications weren't even using lower level primitives like OSSpinLock in the first place, so replacing it was not a concern.
- saagarjha 4y agoWith all due respect, you’re a small indie developer, and the situation is different when you make a browser used by millions of people. Apple does do whatever it wants, but sometimes it (wisely) realizes that what it wants is people to not be upset that people’s favorite app is broken in the next release of macOS, so occasionally steps are taken to ensure that doesn’t happen. This ranges from committing to SPI for longer than it is necessary for Apple’s own use, to putting in bundle ID or linked-on-or-after checks, or in rare cases reaching out to developers directly. The target for this comment is developers like you, but those who are less familiar with the platform than you are but of similar import. Those are the ones who will read this and say “wow Firefox says this is how to make my app fast so I should do it!!1!” and what is going to happen is either they’re going to use it wrong, because they aren’t really experts on how locking works, or they’ll copy the code of of Firefox and their app will break in a future OS release while Mozilla will update their code in July.
- lapcat 4y ago> Those are the ones who will read this and say “wow Firefox says this is how to make my app fast so I should do it!!1!” and what is going to happen is either they’re going to use it wrong, because they aren’t really experts on how locking works, or they’ll copy the code of of Firefox You can't save people from themselves, but in any case this seems to overlook what I said at the end: "the majority of applications weren't even using lower level primitives like OSSpinLock in the first place, so replacing it was not a concern." I wouldn't worry about native Mac developers, who I suspect are mostly a dying breed of grizzled old veterans nowadays. To illustrate, I noticed recently that the last post on https://www.reddit.com/r/macprogramming/ https://www.reddit.com/r/macprogramming/ was over 2 years ago, April 2020.
- ccakes 4y agoCounter argument - https://www.reddit.com/r/swift/ https://www.reddit.com/r/swift/ has posts within the last few hours. Granted Swift is technically a cross platform language but it feels safe to assume that the vast majority of users are developing for macOS and iOS.
- olliej 4y agoHappily, rather than using non-API OS internals, Firefox could recognize that their behaviour does not match normal applications and so they might have to actually implement spin locks themselves rather than relying on a API that is designed to support more general apps. That's what the other browser engines do, including webkit and that's an actual system library on macOS and iOS.
- saagarjha 4y agoYes but also do note that WebKit is probably among the most NIH codebase on the platform.
- olliej 4y agoThose reasons are largely that the things that are perf critical for a browser engine don't match anything that normal apps do. That's the same reason gecko also isn't using the system allocator, as we see here. You really don't want the rest of the OS making the performance trade offs people expect from browser engines these days.
- ben-schaaf 4y agoBlink uses os_unfair_lock on macOS: https://source.chromium.org/chromium/chromium/src/+/main:base/allocator/partition_allocator/spinning_mutex.h https://source.chromium.org/chromium/chromium/src/+/main:bas.... As does WebKit: https://github.com/WebKit/WebKit/blob/4c09a48ea418ff15f65d6c1331dff25ebb4dcf02/Source/bmalloc/libpas/src/libpas/pas_lock.h https://github.com/WebKit/WebKit/blob/4c09a48ea418ff15f65d6c....
- olliej 4y agoYes. Blink and WebKit, and I suspect a bunch of other - correct - software uses os_unfair_lock. Because os_unfair_lock is public, documented, and stable API. The function this article is talking about is not API, it is part of the OS internals, presumably used to implement the standard API functions.
- 4y ago
- olliej 4y agoThe solution here, rather than using OS-internals is for Firefox to implement spin locks themselves, which is the level that they should be thinking if they're making their own allocator. This is what webkit, chromium, etc - webkit even has a blog post: https://webkit.org/blog/6161/locking-in-webkit/ https://webkit.org/blog/6161/locking-in-webkit/
- saagarjha 4y agoI get the impression the author was a little hesitant to do any spinning in userspace.
- ben-schaaf 4y ago> As an intermediate solution, I initially fell back to OSSpinLock on older systems. Later I managed to get rid of it for good by relying on os_unfair_lock plus manual spinning in user-space.
- gsvelto 4y agoYou're confusing general purpose locks with special-purpose ones used in the memory allocator. Firefox has its own GP locks, you can find them here: https://searchfox.org/mozilla-central/search?q=&path=xpcom%2Fthreads%2F*Mutex https://searchfox.org/mozilla-central/search?q=&path=xpcom%2...* However those are not suitable for use in the memory allocator, and neither are WebKit's. WebKit uses os_unfair_lock within its memory allocator: https://github.com/WebKit/WebKit/blob/520379e30f3b2b6d4de9957ab2afc97185879bc4/Source/bmalloc/libpas/src/libpas/pas_lock.h#L120 https://github.com/WebKit/WebKit/blob/520379e30f3b2b6d4de995... And so does Chromium: https://source.chromium.org/chromium/chromium/src/+/main:base/allocator/partition_allocator/spinning_mutex.h;l=54 https://source.chromium.org/chromium/chromium/src/+/main:bas...
- olliej 4y agoCorrect, and that isn't what this article is talking about. This article is about an internal function in the OS that they've decided to is what they'd like to use, and that the only reason they aren't using it is because it doesn't exist in older versions of macOS, because you know, it is an implementation detail of the OS. os_unfair_lock are a documented part of the OS API: https://developer.apple.com/documentation/os/1646466-os_unfair_lock_lock https://developer.apple.com/documentation/os/1646466-os_unfa...