5 ms·
I work for Microsoft. As you can imagine, this is a problem we have devoted a considerable amount of energy to thinking about. A lot of this discussion has revo
by antics 11y ago
I work for Microsoft. As you can imagine, this is a problem we have devoted a considerable amount of energy to thinking about. A lot of this discussion has revolved around comparisons to previous solutions (particularly Cygwin and Interix/SUA), but I think it's worth backing up and thinking about what the fundamental limitations of implementing something approaching POSIX compliance in userspace. I will try to tell the story more or less as it is told internally by the people who built these things from scratch.
To be transparent, I am sort of worried that the authors have told us in this thread that they think the number of applications that "really need" `fork` is small, and that the discussions about POSIX subsystems on Windows are overindexed on discussions about `fork` without justification.
The truth of the matter is that the semantics of `fork` infect every API that creates process state. Every library, every syscall that creates process state will have a clear answer for what happens when a process that is using that bit of process state calls `fork`. This includes file and socket management, IPC stuff, threads, signals, and so on. To make `fork` work properly on Windows, you absolutely need to replicate what UNIX does here, or you will break a lot of apps, because make no mistake, a lot of apps depend on these semantics to work correctly. `fork` is not just a function that occasionally gets called and sometimes needs to be fast for people who are calling it. It is core to the semantics of the POSIX API, and if you don't treat it as such you are in for a bad time later.
Perhaps more worrying, though, is that there is a serious impedence mismatch between the UNIX and Windows models of asynchrony. Particularly in the case of sockets and signals, this difference is immense, and I am extremely skeptical we will find a good (or even close to production-worthy) solution in usermode. Maybe these good folks have found something I have missed; to me the approach just seems doomed.
And of course, this is all just the start of the problem. It is a much longer trudge to solve the "real" problem, which is immense. The original Interix POSIX subsystem in NT 3.5 (I'm told) had a `fork` that was just barely enough to pass the 1003.1-1990 validation suite, and fell over quickly if you pushed much harder. But they didn't have to mess with the `fork` implementation to turn that 67kloc into the more robust and conformant SFU 3.x; it was mostly a long tail of things extrernal to fork that just needed to be ironed out.
On top of that, to really have `fork` interop, you would likely need to redefine every Win32 API so it did the "right thing" under UNIXy things like signal delivery and fork, and that is a truly, terrifyingly tall order.
- dalias 11y agoThese are some good observations. Note that on modern POSIX, calling anything but async-signal-safe functions after forking in a multi-threaded process results in undefined behavior. I think it's totally reasonable to consider any programs using the WinAPI (not just POSIX functions provided through midipix) as being formally multi-threaded, and to consider the WinAPI non-AS-safe. I'm not 100% sure what midipix is doing in this regard, but my advice on the project (I'm the primary author/maintainer of musl libc, which midipix is using) has been not to worry about making arbitrary functions work after fork, but only supporting the things that POSIX requires to work. My view in general is that fork should be phased out, but posix_spawn is not sufficiently powerful yet to replace all uses of fork+exec -- it can't do advanced uid changes, setsid, resource limits, etc., and posix_spawn can't be used effectively from an async signal context because the API (attributes and file actions) inherently involves allocation.
- mpixorg 11y agoThere are two main scenarios for fork(), and the lack of distinction between them is what I suspect led to some of the articulated worries. The first scenario, in the spirit of traditional unix, consists of the sequence fork+execve. This scenraio is supported in midipix, yet requires that the (optional) subsystem performs execve on behalf of the forkee. As noted earlier, if all the child does is call execve, then application authors should consider switching to posix_spawn not only due to its better portability, but also for performance reasons. The second, and arguably more interesting scenario, takes place when the child brnaches away from the parent process, performs (in parallel) a designated portion of a larger task, and then exits. This second scenario has been thoroughly tested and works as expected with respect to IPC, I/O, signal delivery, and nested forking. In either of the above scenarios, there is no expectation that the child could interact with csrss, create or operate on GDI handles, or otherwise use the WIN32 API directly. Then again, it is expected that a process that calls fork() will not have any of the GUI libraries loaded at the time the call is made. Any limitation here, if any, has little to no relevance to existing applications that depend of fork, but only to what I fondly call the "fetishization of fork" in discussions about posix-to-windows portability. Applications that truly depend on fork were written with neither WIN32 nor GDI in mind, and therefore will not break due to such windows-specific interfaces not being available (as an aside, an X11 application calling fork() from within its event loop is an equally bad design regardless of whether the operating system supports that or not). Looking forward to further cooperation! I hope this addresses at least some of the questions you have raised.