4 ms·
The one thing I wasn't sure about was how the per-process global thread pool for non-private doors was populated. I missed it earlier (wasn't looking closely en
by wahern 2y ago
The one thing I wasn't sure about was how the per-process global thread pool for non-private doors was populated. I missed it earlier (wasn't looking closely enough, and the hacks to handle forkall headaches make it look more complicated than it is), but the user space door_create wrapper code--specifically door_create_cmn--invokes door_create_server (through the global door_server_func pointer) on the first non-private door_create call. door_create_server creates a single thread which then calls door_return. On wakeup, before calling the application's handler function, the door_return assembly wrapper conditionally invokes the global door_depletion_cb (defined back in door_calls.c) which can spin up additional threads.
The more I read, the more Doors seems pretty clever. Yes, the secret sauce is very much like scheduler activations or Google's switchto, but you can pass data at the same time, making it useful across processes, not just threads sharing an address space. And you can't get the performance benefit merely by using AF_UNIX sockets because the door_call and door_return have an association for the pendency of the call. The tricky part from an API perspective isn't switching the time slice to the server thread, it's switching it back to the client thread on the return.
Microkernels can do this, of course, but figuring out a set of primitives and an API that fit into the Unix model without introducing anything too foreign, like Mach Ports in macOS, does take some consideration. Now that Linux has process file descriptors, maybe they could be used as an ancillary control message token to get the round-trip time slice and data delivery over AF_UNIX sockets. And memfd objects could be used for messages larger than the receiver's buffer, which seems to be a niche case, anyhow. Or maybe that's over complicating things.
- kragen 2y agoyeah, i've been thinking of doing something like your memfd thing for intranin, the bubbleos kernel, which is still vaporware: transmit one or more pages from the sender to the receiver as part of the ipc, so they disappear from the sender's address space and appear in the receiver's address space. my theory is that, that way, you get the efficiency of shared memory (mostly, at least) without the security risks and debugging headaches. in a repetitive client/server kind of relationship, you can pass the same pages back and forth, so it's effectively just shared memory with exclusive locking, and it extends smoothly to zero-copy networking across hosts, at least if the network message size isn't small compared to a page (you don't want to have to zero a 4096-byte page in order to deliver it up to a user process with a 153-byte message in it) obviously this is the opposite extreme from 'fit into the unix model without introducing anything too foreign' ;) and it may turn out that it's not as good an idea as it seems once i try it. but while some aspects of the unix model are very good, i suspect others were more meritorious on non-networked pdp-11 with respect to starting up a thread to run a callback registered for a message port, i feel like it could be done with a great deal less code than this, though maybe i'm being too optimistic about keeping messy real-world complexity out. in einkornix, which i've only tested as a user-level threading library under linux in emulation, my function for starting up such a thread is einspawn(), which is called (like door_create()) with a callback function and a userdata argument. __door_create() also takes a flags argument; einspawn() instead takes a pointer to the task struct it will finish initializing. it's ten instructions long: http://canonical.org/~kragen/sw/dev3/einkornix.S http://canonical.org/~kragen/sw/dev3/einkornix.S