13 ms·
macOS 13.5 no longer allows setting system wide ulimits
- Angostura 3y agoThe error message indicates that "Operation not permitted while System Integrity Protection is engaged". .... and if SIP is turned off?
- saagarjha 3y agoIt works as expected. $ sudo launchctl limit maxfiles maxfiles 256 unlimited $ sudo launchctl limit maxfiles 65536 200000 $ sudo launchctl limit maxfiles maxfiles 65536 200000
- bombcar 3y agoSo can it be “changed” and then sip turned back on, or does SIP reset it?
- saagarjha 3y agoI think these get reset at reboot?
- bombcar 3y agoThat’s annoying. If you just had to disable SIP and change a value and reenable it wouldn’t be a major issue. I feel Apple is trying to force apps to handle this “the correct way” - remember UAC prompts all over the place when they were first introduced in Windows?
- sgjohnson 3y ago> .... and if SIP is turned off? You lose Touch ID & Apple Pay
- saagarjha 3y agoTouch ID works when SIP is disabled
- macNchz 3y agoUgh, it has been a few years since I did any software development directly within MacOS, but when I did I found it really annoying to have SIP fully enabled. At the time, though, disabling it didn’t take away core features of the computer. The most frustrating thing with SIP was that I’d always butt up against it at the most inopportune moments: deep in a rabbithole of diagnosing some unusual issue, I’d finally have everything running and set up just right to reproduce it, realize I needed to trace a specific process or something, only to have the system tell me I wasn’t allowed to. Perfect timing to have to restart the computer and wait while it slowly boots into recovery, then remember where I left off and recreate my environment. The continuing iOS-ification of MacOS really drove me away from using it as my main computer, despite having been a lifelong Mac user. I still have a Macbook Air, but for any real work it’s just a thin client to my Linux desktop.
- JoBrad 3y agoI’ve slowly moved towards pushing any serious modifications of my system to a container or vm. The cost of messing something up, in terms of lost productivity alone, just isn’t worth it.
- henvic 3y agoGlobals are a nightmare. Despite this being something really common to do for developers and something that will cause a pain for a lot of early adopters of [whatever software that requires it], I see this as a great move!
- crest 3y agoDo you really consider 256 file descriptors a sane resource limit on processes?
- mmis1000 3y agoI hit it because i use ssh to tunnel connections to mac. And sshd seems to use file socket to handle the sessions. I honest don't think it is a sane limit at 2023. Even the most complained 'low performance' platform like nodejs handles thousands of connections just fine. What it the point to have a limit of 256?
- CodesInChaos 3y agoHaving a soft-limit of FD_SETSIZE (typically 1024) prevents memory corruption in applications using `select`. The hard-limit is generally huge (>100k) so applications which don't need that protection can simply raise their soft-limit.
- isodev 3y agoI believe it is given the risks of increasing the limit system-wide. Processes which really require more still have the option to ulimit/setrlimit.
- stephenr 3y agoDo you really think it's sane to write a program that just relies on the user increasing global limits, rather than calling setrlimit appropriately?
- CodesInChaos 3y agoAn application that opens many files and doesn't use `select`, should raise its own soft-limit to match the hard-limit (or a reasonable number below the hard limit, if it wants to self-limit for some reason). The default soft-limit should match FD_SETSIZE and should not be raised globally by the user. I don't know why the default hard limit is 256 and not 1024. Perhaps FD_SETSIZE was lower than 1024 historically?
- kevincox 3y agoThe default limit is in place because select() uses a fixed size bitmap for file descriptors. If a program that uses select opens more than that many files it will make an out of bounds write. It is probably better to make the file descriptor allocation fail than have memory corruption. All programs that don't use select() should raise the limit to the hard limit on startup. Then drop it back down before exec()ing another program. It is a silly dance but that is the cost of legacy. I'm surprised that these programs that are affected can't add this dance in.
- crest 3y agoSelect has 1024 bit/file descriptor bitmap and anyone using it gets what they deserve. MacOS also provides poll() as a slightly less braindead, but still POSIX option. The proper solution is to use kqueue()/kevent() allowing userspace processes to efficiently get notified about changes to tens of thousands of file descriptors (tracking a million file descriptors using this API on FreeBSD works fine).
- kevincox 3y agoI don't know if I would call anyone unfortunate enough to not know about the limitations of select() deserving of such limitations. At least on my Linux box the man page does have a warning at the top of the description section but it is easy to accidentally skim over. It would be interesting to do something like avoid defining that symbol by default, require `-DENABLE_OBSOLETE_SELECT_API` to make it available. It would cause trouble for compiling old software but it is easy to remedy and at least makes new users extra aware that they shouldn't start using this function.
- avhception 3y agoUnfortunately, adding in an option like `-DENABLE_OBSOLETE_SELECT_API` is only trivial if you have a good understanding of the build system in use and sometimes other nuts and bolts of a given project. Tracking down a build failure in some old software project that may or may not have approachable people left on the team through elaborate build scripts that may or may not surface the error in a way that may or may not be easily comprehensible for an outsider can be really hard.
- semireg 3y agoAs an aside, Quinn “The Eskimo!” is a legend. They have helped me on a few occasions with code signing electron apps. In the American south there is a saying “doing the lords work.” Thank goodness we have people like Quinn. Their understanding of complex system and abilities to troubleshoot are invaluable.
- dinkblam 3y agoalso, his posts on the Apple Developer Forums are also much more professional and insightful than the crap which Apple calls their documentation.
- saagarjha 3y agoIt sucks because the forums are one of the worst place he could be putting this information–it's a terrible medium, hard to search, and full of information that ought to be elsewhere. I do not appreciate that Apple can rely on this horrible stopgap instead of writing technotes like they're supposed to.
- nvm0n2 3y agoQuinn does actually author doc pages as well. Some of the code signing tech notes are written by him I think.
- b3morales 3y agoAlso he's got to approaching retirement age at this point, and there is no backbench corps of other DTS who do what he does. Maybe another decade we'll have his help, then what?
- macshome 3y agoI keep an archive and index of his posts on GitHub so that they are easy to find. https://github.com/macshome/The-Wisdom-of-Quinn https://github.com/macshome/The-Wisdom-of-Quinn
- 3y ago
- crest 3y agoAt least it's only the soft limit. The hard limit isn't reduced. The simplest workaround is to wrap the start of the of process with a shell script raising the soft limit before it execs into the application you want to run.
- c0l0 3y agoLet's just hope Apple will not alter the deal any further, right? [-:
- baq 3y agoI've been saying for as long as I've got a macbook pro that the hardware is absolutely above anything available from other vendors but macOS is crap for development. Interesting to know it actively and intentionally becomes even worse :/
- fouc 3y agouse & support asahi linux
- DarkmSparks 3y agoI support asahi linux. But Apple should generally fix this. Having a system wide limit of 256 files open is almost as outdated and idiotic as having a filesystem that doesnt support case or more than 256 characters (NTFS)
- mnd999 3y agoNTFS does support case internally.
- thfuran 3y agoAnd windows now supports longer paths, but I think it's still disabled by default (and I'm sure there's a bunch of legacy software that is still using old APIs and can't support long paths anyways).
- mort96 3y agoWindows Explorer itself is an example of such "legacy software".
- nullindividual 3y agoThe limit was with Win32. It’s not a “legacy” API, but it is as old as NT. There are some applications which do not use the MAX_PATH constant, such as Office (probably one of the few rate examples; Office apps have some horrid hacks like this and how they paint their menu bar). Case sensitivity will break certain applications.
- dinkblam 3y agoa much worse recent change is documented here: https://www.mothersruin.com/software/SuspiciousPackage/faq.html#finder-open-with https://www.mothersruin.com/software/SuspiciousPackage/faq.h... basically no experienced Mac user would ever open a .PKG package with the Installer, but rather open it with a safe inspection app (Suspicious Package, Pacifist) first, either for security inspection of the pre- & post-install scripts, or to do a manual install outright to prevent the security nightmare that actually installing a package is. while inspecting and/or manually installing packages is much safer than installing packages with the installer, it is now effectively outlawed by Apple. continuing to do things in the safe way now takes a lot more time...
- zarzavat 3y agoJust turn off SIP. SIP is for regular users who don’t know what a ulimit is, the whole point of SIP is to lock down the OS as much as possible. If you’re a developer, you live in the Terminal, you obviously need full control over your OS. edit: I appreciate the irony of being downvoted for suggesting having control over your OS on Hacker News, so keep it coming please. Mo’ downvotes mo’ irony.
- baq 3y agolaughs in corporate IT
- sspiff 3y agoI'm a software developer at a company where ~50% of the staff is a developer and our IT fleet management enforces SIP. This simply isn't an option for us because of security requirements from our customers.
- zarzavat 3y agoOK, but this is not macOS’s fault.
- sspiff 3y agoSIP is the default though. Turning it off is an option for some, for now. But Apple has clearly indicated they're moving towards a more restrictive ecosystem through there actions for a long time now.
- DannyBee 3y agoIt turns out you get to take responsibility for the downstream consequences of your actions, whether you expected them or others help you or not. That is true in most social, legal, etc settings. (in law, it's known as but-for causation) So yes, this is MacOS's fault.
- tinus_hn 3y agoTry moving to Windows and seeing how free you are to setup things in an enterprise environment.
- amelius 3y agoOnly superroot (i.e., Apple) can change it.
- CodesInChaos 3y agoI think a security option that does the following would make sense: 1. exec should reduce the hard-limit to FD_SETSIZE unless it's passed a flag 2. When a process calls `select` while having a hard-limit > FD_SETSIZE, it gets terminated. (Strictly speaking this doesn't prevent the memory corruption, since it happens in select's support macros, not select itself. But I'd expect it to be good enough in practice) A modern application which doesn't use `select` should raise its own hard-limit during startup, option into the select-termination in exchange for the ability to open many files.
- roblh 3y agoI am, at this exact moment, getting bodied by a ulimit problem on mac. Apparently with pacman, directories attached with bind mounts have a fixed ulimit of 64 internally and running npm install inside a mounted project explodes completely because of it. Funny that this turns up right now, even if it’s not a fix for my particular problem.
- c-hendricks 3y agoI've ... always used ulimit on macOS, going back almost a decade. Never heard of using `launchctl` to change it. https://www.google.com/search?q=ulimit+mac+site%3Astackoverflow.com&oq=ulimit+mac+site%3Astackoverflow.com https://www.google.com/search?q=ulimit+mac+site%3Astackoverf... vs https://www.google.com/search?q=%22launchctl+limit+maxfiles%22+site%3Astackoverflow.com https://www.google.com/search?q=%22launchctl+limit+maxfiles%...
- pierat 3y agoSo, who's computer is it REALLY when you can't make settings/decisions about the computer you bought? When some other entity, manufacturer included, is making decisions you cant change or reverse, sounds like they retain real ownership. Its high time for the FTC to start busting fraudulent rentals misrepresented as a "sale". (And anybody viewing the Apple ecosystem KNEW this anti-owner lockdown phone shit would eventually hit their laptop and desktop computers as well. But for some reason, the Apple fanatics are OK with this stuff. )
- donatj 3y agoYou own the hardware. You license the software. That relationship is pretty clear.
- pierat 3y agoAnd there's your problem right there. Intellectual property rights (copyright, trademark, patent) should NOT dilute or devalue or remove inherent ownership from property rights. And "license" presented after the purchase fact should make that the equivalent of toilet paper. It's a "license" after they have your money and over a barrel. There's no agreeing to that. At at least the Europeans with the GDPR realize that farcical situation and illegalized it appropriately. "Agree or turn your $X000 device you paid for into a paperweight" is no agreement.