10 ms·

Why Ruby App Servers Break on MacOS High Sierra

259 points by mef 9y ago

joshmn 9y ago
This issue has been addressed on ruby-head https://github.com/ruby/ruby/commit/8b182a7f7d798ab6539518fbfcb51c78549f9733 https://github.com/ruby/ruby/commit/8b182a7f7d798ab6539518fb...
- akvadrako 9y ago
  It's been addressed, but it's a hack. To be safe, apps must stop all other threads before forking. The only correct fix I can see is to have pre- and post- fork hooks in every library, which make that guarantee.
  - nateberkopec 9y ago
    In Puma, we emit a warning if we detect any additional threads running prior to fork.
- jasonmp85 9y ago
  Right; this article is almost a week old at this point and some progress has been made in addressing this. In particular, I found the characterization that other app servers are not taking action on this problem to be a lie in poor taste. This kind of talking up of one's own product and saying "we're first; we're best!" rather than working within the ecosystem to help fix a shared issue confirms my feeling that Passenger is a bit of an outlier in the Ruby community.
oblio 9y ago
This is so incredibly Apple :) The breakage, I mean. To clarify a bit, for better or for worse, this is what Microsoft does, totally different psychology: https://blogs.msdn.microsoft.com/oldnewthing/20031223-00/?p=41373 https://blogs.msdn.microsoft.com/oldnewthing/20031223-00/?p=...
- mrpippy 9y ago
  Apple isn’t breaking old/existing binaries, the new behavior only applies to binaries compiled against the 10.13 SDK.
  - gradstudent 9y ago
    is that so much better? I dread every new release of macos because it always breaks some part of my toolchain (either gcc and friends, or valgrind or whatever). It wasn't so bad when a new os came every few years. now they break stuff every year. six months since I bought a new machine and my gdb still isn't working quite right.
    - sigjuice 9y ago
      Apple is not responsible for every piece of third party software, especially niche tools like gcc, gdb and valgrind. It is up to the maintainers of those tools to keep up with new macOS releases. Beta releases of macOS are made available early to developers for exactly this reason.
      - familyit 9y ago
        You work there for sure
        dang 9y ago
        This crosses into personal attack, which is a bannable offense on HN. Would you please read https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and post only civil, substantive comments from now on?
      - realusername 9y ago
        gcc isn't exactly a "niche tool".
        9y ago
the_mitsuhiko 9y ago
Python suffers from the same issue but the response there has largely been to stop using modules that use objc.
- captainmuon 9y ago
  I was thinking about Python when reading this, the real culprit is not objc but the third-party modules. If I understand correctly, this problem is caused when you call initialize in another thread, and while that is running, you fork. It affects these app servers because this is triggered at "require" (or "import") time. This is madness, no module should run code just when you import it. OK, I have been guilty of that, too. But if absolutely neccessary, keep it to a minimum. It breaks all kind of things when module imports are not side-effect-free. Launching a thread and (indirectly) taking a mutex is definitely not something you should do on module import!
  - dchest 9y ago
    As far as I understood, the module isn't launching threads at import time. On the contrary, the problem is that since pg is linked against macOS's Kerberos and LDAP frameworks, as soon as the Objective-C runtime realizes that NSPlaceholderDictionary initialize method was called after fork(), it crashes: One of the rules that Apple defined is that you may not call Foundation class initializers after forking.
    - captainmuon 9y ago
      In that case, the new behavior doesn't make sense. You could call `fork` as the very first thing in a program, and then do all the stuff in the subprocess, and I wouldn't expect it to make a big difference. Maybe there is an internal `mark_exec_as_called()` function that you could exploit, if you are forking without exec...
    - eridius 9y ago
      Well, something must be spawning a thread first. From the Obj-C source¹, the comment on performForkChildInitialize() (the point where this crash happens) says > Exception: processes that are single-threaded when fork() is called have no restrictions on +initialize in the child. Examples: sshd and httpd. ¹ https://opensource.apple.com/source/objc4/objc4-723/runtime/objc-initialize.mm.auto.html https://opensource.apple.com/source/objc4/objc4-723/runtime/...
pantulis 9y ago
I love this geek-porn stuff, and the Phusion guys never fail to deliver it ;) But my question is: is this really that important? I mostly use macOS for development, don't feel that the preforking model has that impact in the development cycle.
- digger250 9y ago
  You are not affected, but there are people in the world besides yourself who do things differently than you.
- FooBarWidget 9y ago
  It is important for dev-prod parity. You will want to test whether your code is compatible with preforking, which you will likely use in production.
lima 9y ago
Why do people run applications servers on macOS?
- bantunes 9y ago
  Maybe they're developers running a local copy of the projects they're working on?
- geocar 9y ago
  The underlying problem also affects Linux. Apple is probably tired of bug reports to their software that really stem from some other place.
- tonyedgecombe 9y ago
  https://www.apple.com/uk/macos/server/ https://www.apple.com/uk/macos/server/
  - giancarlostoro 9y ago
    I've mostly seen it for managing multiple Macs in a classroom environment. Not sure I've seen it used heavily in production to run web servers though. Was curious and was able to find sites that do let you rent Macs that you remote into, interesting setup since you know exactly the hardware you're getting into.
mhandley 9y ago
Interesting discussion. If these were user-space threads, like FreeBSD ~20 years ago, there'd be no problem. When fork() is called, the whole user-space threads package would be forked, along with all the threads. So the obvious question is whether it's fundamental that with kernel threads the fork() system call doesn't clone all the other threads in the process? Yes, that's not how it's done, but could Apple choose to implement a new fork_all() system call? I imagine it wouldn't be easy - you'd need to pause all the running threads while you copied state, but is there a reason it's actually not possible?
- mhandley 9y ago
  Thinking about it some more, one problem would be what to do with threads that have just called a system call and are waiting for a response. To have both parent and child process system calls complete without errors would require cloning the kernel state, and that's not trivial. It some cases, there's probably no good answer, short of causing one of the two system calls to return an error, and that in itself would confuse a lot of user-space software. I suppose you could return something similar to EAGAIN, but you'd be dependent on user-space handling that cleanly. Maybe you could hide this under libc?
  - wruza 9y ago
    EINTR? I suspect some calls are defined to never return it, but it may be just de facto, not strict rule. Edit: simply signalling all threads with SIGFORK, waiting for a handler to return and freezing a thread at SA_RESTART point feels right way to do it. Why did POSIX decide to fork() only one thread by default at all?
    - crest 9y ago
      Because the alternatives are worse.
  - ScottBurson 9y ago
    So this is a worse-is-better design decision driven by a previous worse-is-better design decision. If system calls were interruptible, then the problem of threads in system calls would go away.
krisives 9y ago
What does Linux do right now?
- dchest 9y ago
  If you fork after launching threads, your program is an unexpected state, so it may corrupt data or crash. The same happens on macOS, it's just that Apple added a contract that you can't initialize ObjC classes after forking.
throwaway613834 9y ago
fork() fundamentally does not make sense as the de-facto method of starting a new process. Why aren't people using posix_spawn() by default?
- tedunangst 9y ago
  Because that doesn't share memory.
  - twic 9y ago
    You can share memory in other ways, via mmap or SysV shared memory. You could imagine an implementation of the Ruby interpreter which kept all the expensive and shareable state (bytcode and class metadata, i suppose) in a shared memory segment, and used spawning to create worker processes with their own process heaps, but sharing that segment. It would be a lot of work to implement. It would probably be easier just to get Ruby to use threads properly.
    - bpicolo 9y ago
      > bytcode and class metadata There's a bit more to it than that, as programs can run code and load data prefork. Pretty important for things like large config files.
- segmondy 9y ago
  implementation of fork is defined by the POSIX standards. Apple should respect it. There are tons of apps that have been written using fork, are you going to change all of them to use posix_spawn()?
  - deleted 9y ago
    [deleted]
  - tedunangst 9y ago
    Posix standard says you can't call anything but async functions after fork() in a threaded program. Application developers should respect that.
saagarjha 9y ago
Greg Parker, who works on the Objective-C runtime, has a blog post that goes into more detail: http://www.sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html http://www.sealiesoftware.com/blog/archive/2017/6/5/Objectiv...
- hinkley 9y ago
  So, basically Objective-C doesn't like the RAII pattern if the resource is a process? Ouch. I got the gist of his summary but his writing is a bit... awkward. Is English not his first language?
  - saagarjha 9y ago
    > Is English not his first language? As far as I know it is.
  - valleyer 9y ago
    C has this same problem. Most of the C standard library is unsafe to use between fork and exec of a multithreaded program. This usually includes malloc and printf.
    - dozzie 9y ago
      How so? After fork() you are guaranteed to only have one remaining thread in the child process, and the parent process doesn't care if there was a fork() or not.
      - valleyer 9y ago
        malloc() and printf(), for example, have mutexes internal to their implementation. Suppose the parent process has two threads. One is about to do a fork, and the other is in the middle of a malloc(). The fork occurs. The parent process continues on as normal. In the child process, there is only one thread -- a clone of the forking thread, but the memory state of the child process is a full clone of the parent (except for the return value of fork(), of course). The single thread in the child calls malloc(). But the malloc mutex is already held, because in the parent process, a thread was executing there. Unfortunately, since that thread does not exist in the child, it will never be able to release the mutex. The thread in the child is deadlocked.
ransom1538 9y ago
I agree the bug should be fixed. But, why not just use docker, then run rails like its on ubuntu/linux on your mac? It miserable having windows/mac/etc specific issues.
- sillysaurus3 9y ago
  Docker grows to >8GB. I have 13GB free, 486GB used. My free space fluctuates by as much as 20GB depending on how much RAM I'm using. (And by "I'm" I mean "Chrome.") I've never felt the need to install docker on my local dev environment. It's great for production, and I'm sure it's great for people who can afford the disk space. But when space became tight, Docker was the first to get the axe. I haven't missed it yet.
  - jasonjei 9y ago
    I'm just curious how you would deploy on a container production environment if you can't test all the platform-specific issues while developing on Mac instead of using Docker to develop for a container platform. Isn't the whole point of using Docker to minimize the differences between production and development?
    - matwood 9y ago
      Deploy to a container test environment first? I hope no one is going from development -> production.
- deleted 9y ago
  [deleted]
throwme211345 9y ago
LOL. mac osx breaks fork() to avoid state inconsistency in threaded applications. How about pthread_atfork() semantics? But ,as usual, apple heavy hands userspace and breaks things. Nothing new to see here, move on.
- throwme211345 9y ago
  What you don't like the fact that apple sucks for breaking userspace (as usual) or that pthread_atfork() type approaches should be in every programmers toolbox?
  - oshepherd 9y ago
    POSIX has deprecated pthread_atfork because it is unworkable. In particular, atfork handlers can only call AS-safe functions, which means they're useless.
    - throwme211345 9y ago
      As you say except I explicitly noted '..type approaches' and '..semantics'. If a library designer does things in a way that makes you doubtful of state then don't fork. If you do fork block signals and exec. It doesn't help the race but it does help your peace of mind (i did what i could). Apple still sucks BTW. Heavy handed nonsense. Let developers deal with the consequences of their actions.
mkj 9y ago
Do developers just call whatever function seems to work without reading the docs? It doesn't work for low level programming. ~ man fork CAVEATS: There are limits to what you can do in the child process. To be totally safe you should restrict yourself to only executing async-signal safe operations until such time as one of the exec functions is called. All APIs, including global data symbols, in any framework or library should be assumed to be unsafe after a fork() unless explicitly documented...
- tedunangst 9y ago
  Yes. Programming by coincidence is the norm. Seems to work, must be right.
- Dangeranger 9y ago
  My suspicion is that this behavior is left over from early implementations of Ruby where threading wasn’t as common and so wasn’t as large of a concern.
- ProAm 9y ago
  Welcome to the stackoverflow era of programming, where technical specs don't matter and frameworks are life.
  - mrguyorama 9y ago
    This has been around since well before stack overflow Here are a plethora of examples from the Windows 95 days: http://ptgmedia.pearsoncmg.com/images/9780321440303/samplechapter/Chen_bonus_ch02.pdf http://ptgmedia.pearsoncmg.com/images/9780321440303/samplech...
- viraptor 9y ago
  It's silly to point at this documentation as if this is something that people will read in this case. People, or specifically web developers are not calling fork(). What they're doing is using supported language, a webapp server, using a database access library, and enabling the preloading option. As seen in the bug report in Ruby, this comes down to coordination between those projects. Each one is correct on their own. Putting them together as a web developer shouldn't involve analysing syscalls you're going to make either.
booleanbetrayal 9y ago
for other fun low-level High Sierra issues, see the PostgreSQL msync() thread: https://www.postgresql.org/message-id/flat/13746.1506974083%40sss.pgh.pa.us#13746.1506974083@sss.pgh.pa.us https://www.postgresql.org/message-id/flat/13746.1506974083%...
- salarycommenter 9y ago
  Interesting. Mapping 64k at a time seems a bit excessive. If I call mmap it's to map everything I'm going to need, but maybe I am missing context.
  - snuxoll 9y ago
    PostgreSQL still supports 32-bit platforms, heap and index files can get large enough that it's not feasible to mmap them into a 32-bit process. As far as the specific number of 64k though, I don't know why they choose that, writing 8 pages to disk at a time (especially sequentially) doesn't really take advantage of modern hardware well.
bpicolo 9y ago
I've hit similar issues with uwsgi in recent memory (though pre high-sierra), where an OS upgrade caused it to start segfaulting when using the `requests` lib inside CoreFoundation somewhere (though of course entirely unrelated to the new forking changes). Maybe this? Though the resolution was to disable uwsgi proxying globally... https://stackoverflow.com/questions/35650520/uwsgi-segmentation-fault-when-using-flask-and-python-requests https://stackoverflow.com/questions/35650520/uwsgi-segmentat...
Girlang 9y ago
It works fine on Windows 10. Stop using an OS designed in 1969.
olivierlacan 9y ago
The discussion on the Ruby core team issue tracker is also very informative: https://bugs.ruby-lang.org/issues/14009 https://bugs.ruby-lang.org/issues/14009
dep_b 9y ago
> This cryptic error Well....I don't think I've seen many programmer-to-programmer errors that are less cryptic than the one described in the article. It's actually quite amazing how much explanation you sometimes get from Cocoa!
- LeoNatan25 9y ago
  I think it's mostly unclear to non-Apple developers.
- semanticfact 9y ago
  It's only cryptic if you don't read it, right?!?

259 points by mef 9y ago

This issue has been addressed on ruby-head https://github.com/ruby/ruby/commit/8b182a7f7d798ab6539518fbfcb51c78549f9733 https://github.com/ruby/ruby/commit/8b182a7f7d798ab6539518fb...

akvadrako 9y ago
It's been addressed, but it's a hack. To be safe, apps must stop all other threads before forking. The only correct fix I can see is to have pre- and post- fork hooks in every library, which make that guarantee.
- nateberkopec 9y ago
  In Puma, we emit a warning if we detect any additional threads running prior to fork.
jasonmp85 9y ago
Right; this article is almost a week old at this point and some progress has been made in addressing this. In particular, I found the characterization that other app servers are not taking action on this problem to be a lie in poor taste. This kind of talking up of one's own product and saying "we're first; we're best!" rather than working within the ecosystem to help fix a shared issue confirms my feeling that Passenger is a bit of an outlier in the Ruby community.

oblio 9y ago

This is so incredibly Apple :) The breakage, I mean. To clarify a bit, for better or for worse, this is what Microsoft does, totally different psychology: https://blogs.msdn.microsoft.com/oldnewthing/20031223-00/?p=41373 https://blogs.msdn.microsoft.com/oldnewthing/20031223-00/?p=...

mrpippy 9y ago
Apple isn’t breaking old/existing binaries, the new behavior only applies to binaries compiled against the 10.13 SDK.
- gradstudent 9y ago
  is that so much better? I dread every new release of macos because it always breaks some part of my toolchain (either gcc and friends, or valgrind or whatever). It wasn't so bad when a new os came every few years. now they break stuff every year. six months since I bought a new machine and my gdb still isn't working quite right.
  - sigjuice 9y ago
    Apple is not responsible for every piece of third party software, especially niche tools like gcc, gdb and valgrind. It is up to the maintainers of those tools to keep up with new macOS releases. Beta releases of macOS are made available early to developers for exactly this reason.
    - familyit 9y ago
      You work there for sure
      - dang 9y ago
        This crosses into personal attack, which is a bannable offense on HN. Would you please read https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and post only civil, substantive comments from now on?
    - realusername 9y ago
      gcc isn't exactly a "niche tool".
      - 9y ago

the_mitsuhiko 9y ago

Python suffers from the same issue but the response there has largely been to stop using modules that use objc.

captainmuon 9y ago
I was thinking about Python when reading this, the real culprit is not objc but the third-party modules. If I understand correctly, this problem is caused when you call initialize in another thread, and while that is running, you fork. It affects these app servers because this is triggered at "require" (or "import") time. This is madness, no module should run code just when you import it. OK, I have been guilty of that, too. But if absolutely neccessary, keep it to a minimum. It breaks all kind of things when module imports are not side-effect-free. Launching a thread and (indirectly) taking a mutex is definitely not something you should do on module import!
- dchest 9y ago
  As far as I understood, the module isn't launching threads at import time. On the contrary, the problem is that since pg is linked against macOS's Kerberos and LDAP frameworks, as soon as the Objective-C runtime realizes that NSPlaceholderDictionary initialize method was called after fork(), it crashes: One of the rules that Apple defined is that you may not call Foundation class initializers after forking.
  - captainmuon 9y ago
    In that case, the new behavior doesn't make sense. You could call `fork` as the very first thing in a program, and then do all the stuff in the subprocess, and I wouldn't expect it to make a big difference. Maybe there is an internal `mark_exec_as_called()` function that you could exploit, if you are forking without exec...
  - eridius 9y ago
    Well, something must be spawning a thread first. From the Obj-C source¹, the comment on performForkChildInitialize() (the point where this crash happens) says > Exception: processes that are single-threaded when fork() is called have no restrictions on +initialize in the child. Examples: sshd and httpd. ¹ https://opensource.apple.com/source/objc4/objc4-723/runtime/objc-initialize.mm.auto.html https://opensource.apple.com/source/objc4/objc4-723/runtime/...

pantulis 9y ago

I love this geek-porn stuff, and the Phusion guys never fail to deliver it ;) But my question is: is this really that important? I mostly use macOS for development, don't feel that the preforking model has that impact in the development cycle.

digger250 9y ago
You are not affected, but there are people in the world besides yourself who do things differently than you.
FooBarWidget 9y ago
It is important for dev-prod parity. You will want to test whether your code is compatible with preforking, which you will likely use in production.

lima 9y ago

Why do people run applications servers on macOS?

bantunes 9y ago
Maybe they're developers running a local copy of the projects they're working on?
geocar 9y ago
The underlying problem also affects Linux. Apple is probably tired of bug reports to their software that really stem from some other place.
tonyedgecombe 9y ago
https://www.apple.com/uk/macos/server/ https://www.apple.com/uk/macos/server/
- giancarlostoro 9y ago
  I've mostly seen it for managing multiple Macs in a classroom environment. Not sure I've seen it used heavily in production to run web servers though. Was curious and was able to find sites that do let you rent Macs that you remote into, interesting setup since you know exactly the hardware you're getting into.

mhandley 9y ago

Interesting discussion. If these were user-space threads, like FreeBSD ~20 years ago, there'd be no problem. When fork() is called, the whole user-space threads package would be forked, along with all the threads. So the obvious question is whether it's fundamental that with kernel threads the fork() system call doesn't clone all the other threads in the process? Yes, that's not how it's done, but could Apple choose to implement a new fork_all() system call? I imagine it wouldn't be easy - you'd need to pause all the running threads while you copied state, but is there a reason it's actually not possible?

mhandley 9y ago
Thinking about it some more, one problem would be what to do with threads that have just called a system call and are waiting for a response. To have both parent and child process system calls complete without errors would require cloning the kernel state, and that's not trivial. It some cases, there's probably no good answer, short of causing one of the two system calls to return an error, and that in itself would confuse a lot of user-space software. I suppose you could return something similar to EAGAIN, but you'd be dependent on user-space handling that cleanly. Maybe you could hide this under libc?
- wruza 9y ago
  EINTR? I suspect some calls are defined to never return it, but it may be just de facto, not strict rule. Edit: simply signalling all threads with SIGFORK, waiting for a handler to return and freezing a thread at SA_RESTART point feels right way to do it. Why did POSIX decide to fork() only one thread by default at all?
  - crest 9y ago
    Because the alternatives are worse.
- ScottBurson 9y ago
  So this is a worse-is-better design decision driven by a previous worse-is-better design decision. If system calls were interruptible, then the problem of threads in system calls would go away.

krisives 9y ago

What does Linux do right now?

dchest 9y ago
If you fork after launching threads, your program is an unexpected state, so it may corrupt data or crash. The same happens on macOS, it's just that Apple added a contract that you can't initialize ObjC classes after forking.

throwaway613834 9y ago

fork() fundamentally does not make sense as the de-facto method of starting a new process. Why aren't people using posix_spawn() by default?

tedunangst 9y ago
Because that doesn't share memory.
- twic 9y ago
  You can share memory in other ways, via mmap or SysV shared memory. You could imagine an implementation of the Ruby interpreter which kept all the expensive and shareable state (bytcode and class metadata, i suppose) in a shared memory segment, and used spawning to create worker processes with their own process heaps, but sharing that segment. It would be a lot of work to implement. It would probably be easier just to get Ruby to use threads properly.
  - bpicolo 9y ago
    > bytcode and class metadata There's a bit more to it than that, as programs can run code and load data prefork. Pretty important for things like large config files.
segmondy 9y ago
implementation of fork is defined by the POSIX standards. Apple should respect it. There are tons of apps that have been written using fork, are you going to change all of them to use posix_spawn()?
- deleted 9y ago
  [deleted]
- tedunangst 9y ago
  Posix standard says you can't call anything but async functions after fork() in a threaded program. Application developers should respect that.

saagarjha 9y ago

Greg Parker, who works on the Objective-C runtime, has a blog post that goes into more detail: http://www.sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html http://www.sealiesoftware.com/blog/archive/2017/6/5/Objectiv...

hinkley 9y ago
So, basically Objective-C doesn't like the RAII pattern if the resource is a process? Ouch. I got the gist of his summary but his writing is a bit... awkward. Is English not his first language?
- saagarjha 9y ago
  > Is English not his first language? As far as I know it is.
- valleyer 9y ago
  C has this same problem. Most of the C standard library is unsafe to use between fork and exec of a multithreaded program. This usually includes malloc and printf.
  - dozzie 9y ago
    How so? After fork() you are guaranteed to only have one remaining thread in the child process, and the parent process doesn't care if there was a fork() or not.
    - valleyer 9y ago
      malloc() and printf(), for example, have mutexes internal to their implementation. Suppose the parent process has two threads. One is about to do a fork, and the other is in the middle of a malloc(). The fork occurs. The parent process continues on as normal. In the child process, there is only one thread -- a clone of the forking thread, but the memory state of the child process is a full clone of the parent (except for the return value of fork(), of course). The single thread in the child calls malloc(). But the malloc mutex is already held, because in the parent process, a thread was executing there. Unfortunately, since that thread does not exist in the child, it will never be able to release the mutex. The thread in the child is deadlocked.

ransom1538 9y ago

I agree the bug should be fixed. But, why not just use docker, then run rails like its on ubuntu/linux on your mac? It miserable having windows/mac/etc specific issues.

sillysaurus3 9y ago
Docker grows to >8GB. I have 13GB free, 486GB used. My free space fluctuates by as much as 20GB depending on how much RAM I'm using. (And by "I'm" I mean "Chrome.") I've never felt the need to install docker on my local dev environment. It's great for production, and I'm sure it's great for people who can afford the disk space. But when space became tight, Docker was the first to get the axe. I haven't missed it yet.
- jasonjei 9y ago
  I'm just curious how you would deploy on a container production environment if you can't test all the platform-specific issues while developing on Mac instead of using Docker to develop for a container platform. Isn't the whole point of using Docker to minimize the differences between production and development?
  - matwood 9y ago
    Deploy to a container test environment first? I hope no one is going from development -> production.
deleted 9y ago
[deleted]

throwme211345 9y ago

LOL. mac osx breaks fork() to avoid state inconsistency in threaded applications. How about pthread_atfork() semantics? But ,as usual, apple heavy hands userspace and breaks things. Nothing new to see here, move on.

throwme211345 9y ago
What you don't like the fact that apple sucks for breaking userspace (as usual) or that pthread_atfork() type approaches should be in every programmers toolbox?
- oshepherd 9y ago
  POSIX has deprecated pthread_atfork because it is unworkable. In particular, atfork handlers can only call AS-safe functions, which means they're useless.
  - throwme211345 9y ago
    As you say except I explicitly noted '..type approaches' and '..semantics'. If a library designer does things in a way that makes you doubtful of state then don't fork. If you do fork block signals and exec. It doesn't help the race but it does help your peace of mind (i did what i could). Apple still sucks BTW. Heavy handed nonsense. Let developers deal with the consequences of their actions.

mkj 9y ago

Do developers just call whatever function seems to work without reading the docs? It doesn't work for low level programming. ~ man fork CAVEATS: There are limits to what you can do in the child process. To be totally safe you should restrict yourself to only executing async-signal safe operations until such time as one of the exec functions is called. All APIs, including global data symbols, in any framework or library should be assumed to be unsafe after a fork() unless explicitly documented...

tedunangst 9y ago
Yes. Programming by coincidence is the norm. Seems to work, must be right.
Dangeranger 9y ago
My suspicion is that this behavior is left over from early implementations of Ruby where threading wasn’t as common and so wasn’t as large of a concern.
ProAm 9y ago
Welcome to the stackoverflow era of programming, where technical specs don't matter and frameworks are life.
- mrguyorama 9y ago
  This has been around since well before stack overflow Here are a plethora of examples from the Windows 95 days: http://ptgmedia.pearsoncmg.com/images/9780321440303/samplechapter/Chen_bonus_ch02.pdf http://ptgmedia.pearsoncmg.com/images/9780321440303/samplech...
viraptor 9y ago
It's silly to point at this documentation as if this is something that people will read in this case. People, or specifically web developers are not calling fork(). What they're doing is using supported language, a webapp server, using a database access library, and enabling the preloading option. As seen in the bug report in Ruby, this comes down to coordination between those projects. Each one is correct on their own. Putting them together as a web developer shouldn't involve analysing syscalls you're going to make either.

booleanbetrayal 9y ago

for other fun low-level High Sierra issues, see the PostgreSQL msync() thread: https://www.postgresql.org/message-id/flat/13746.1506974083%40sss.pgh.pa.us#13746.1506974083@sss.pgh.pa.us https://www.postgresql.org/message-id/flat/13746.1506974083%...

salarycommenter 9y ago
Interesting. Mapping 64k at a time seems a bit excessive. If I call mmap it's to map everything I'm going to need, but maybe I am missing context.
- snuxoll 9y ago
  PostgreSQL still supports 32-bit platforms, heap and index files can get large enough that it's not feasible to mmap them into a 32-bit process. As far as the specific number of 64k though, I don't know why they choose that, writing 8 pages to disk at a time (especially sequentially) doesn't really take advantage of modern hardware well.

bpicolo 9y ago

I've hit similar issues with uwsgi in recent memory (though pre high-sierra), where an OS upgrade caused it to start segfaulting when using the `requests` lib inside CoreFoundation somewhere (though of course entirely unrelated to the new forking changes). Maybe this? Though the resolution was to disable uwsgi proxying globally... https://stackoverflow.com/questions/35650520/uwsgi-segmentation-fault-when-using-flask-and-python-requests https://stackoverflow.com/questions/35650520/uwsgi-segmentat...

Girlang 9y ago

It works fine on Windows 10. Stop using an OS designed in 1969.

olivierlacan 9y ago

The discussion on the Ruby core team issue tracker is also very informative: https://bugs.ruby-lang.org/issues/14009 https://bugs.ruby-lang.org/issues/14009

dep_b 9y ago

> This cryptic error Well....I don't think I've seen many programmer-to-programmer errors that are less cryptic than the one described in the article. It's actually quite amazing how much explanation you sometimes get from Cocoa!

LeoNatan25 9y ago
I think it's mostly unclear to non-Apple developers.
semanticfact 9y ago
It's only cryptic if you don't read it, right?!?