4 ms·

What's wrong with sockets? They have quite a few advantages for IPC over Unix signals. - They're scoped within the program (no global signal handler that you n

by duped 3y ago

What's wrong with sockets? They have quite a few advantages for IPC over Unix signals. - They're scoped within the program (no global signal handler that you need to trampoline out of) - The number of IPC channels you can create is effectively unbounded (not really true, but much less limiting than signals). If another process or part of the program needs IPC you can just open a new channel without breaking code or invariants relied upon by any other IPC channels. - Reads/writes can be handled asynchronously without interrupting any thread in the program. - You can use them across the network (AF_INET), VMs (AF_VSOCK), or restrict locally (AF_UNIX) - Unix sockets can be used to send file descriptors around, even if they're opened by a child after forking. That includes using Unix sockets to send the file descriptor of other sockets (eg program 1 is talking to program 2 on the same machine and program 2 opens an IPC channel with program 3 and wants to send it back to program 1). I feel like just because you can use signals for IPC doesn't mean you should.

ajross 3y ago
So, nothing is wrong with sockets. Sockets (both unix domain and TCP) are the overwhelming choice for IPC mechanisms in new code, have been for the last few decades, and will be when we all retire. Nonetheless it's not uncommon to have a communication style where (1) messages are extremely simple, with either no metadata or just a single number associated with them, (2) may be sent to any of a large number of receiving processes. And there, signals have a few advantages: you don't need to manage a separate file descriptor per receiver, you don't need to write a full protocol parser, you can test or trigger your API from the command line with e.g. /usr/bin/kill. They're good stuff. But do be aware of the inherent synchronization problems with traditional handlers. Signals interrupt the target's code and can't block to wait for it (they act like "interrupts" in that sense), so traditional synchronization strategies don't work and there are traps everywhere. If you're writing new signal code you really want to be using signalfd (which, yeah, re-introduces the "one extra file descriptor per receiver" issue).
- duped 3y ago
  Using a signal handler for IPC isn't really "simple" though, since the handler needs to be async signal safe itself. You don't need a "full protocol parser" for sockets either. You can send/receive C structs just fine. It's also not hard to write to sockets from the command line. > But do be aware of the inherent synchronization problems with traditional handlers. This is why you should almost never use signal handlers for IPC, because they're full of footguns and not actually simple to use. Increasing the number of file descriptors doesn't seem like much of a burden. If your app actually pushes on those limits you need to be messing with ulimits anyway.
  - ajross 3y ago
    > You don't need a "full protocol parser" for sockets either. You can send/receive C structs just fine. It's also not hard to write to sockets from the command line. Yeah, that's how you get vulnerabilities. For the same reason that "Thou shalt not write async signal handlers without extreme care", thou must never send data down a socket without a clear specification for communication. It doesn't (and shouldn't) have to be a "complicated" protocol. But it absolutely must be a fully specified one. And signals, as mentioned, don't really have that shortcoming due to their extreme simplicity. They're just interrupts and don't pass data (except in the sense that they cause the target to re-inspect state, etc...). They're a tool in the box and you should know how they work. Mostly the point is just "IPC is hard", and choosing tools isn't really the hard part. Signals have their place.
deleted 3y ago
[deleted]
goodyduru 3y ago
Author here. I agree with you on preferring sockets to signals for IPC. It's more straightforward and stable to use it. This article is one of a series I'm writing on IPC. Writing about them ensures I understand them better. I wrote one on sockets[1]. [1] https://goodyduru.github.io/os/2023/10/03/ipc-unix-domain-sockets.html https://goodyduru.github.io/os/2023/10/03/ipc-unix-domain-so...

by duped 3y ago

ajross 3y ago

So, nothing is wrong with sockets. Sockets (both unix domain and TCP) are the overwhelming choice for IPC mechanisms in new code, have been for the last few decades, and will be when we all retire. Nonetheless it's not uncommon to have a communication style where (1) messages are extremely simple, with either no metadata or just a single number associated with them, (2) may be sent to any of a large number of receiving processes. And there, signals have a few advantages: you don't need to manage a separate file descriptor per receiver, you don't need to write a full protocol parser, you can test or trigger your API from the command line with e.g. /usr/bin/kill. They're good stuff. But do be aware of the inherent synchronization problems with traditional handlers. Signals interrupt the target's code and can't block to wait for it (they act like "interrupts" in that sense), so traditional synchronization strategies don't work and there are traps everywhere. If you're writing new signal code you really want to be using signalfd (which, yeah, re-introduces the "one extra file descriptor per receiver" issue).

duped 3y ago
Using a signal handler for IPC isn't really "simple" though, since the handler needs to be async signal safe itself. You don't need a "full protocol parser" for sockets either. You can send/receive C structs just fine. It's also not hard to write to sockets from the command line. > But do be aware of the inherent synchronization problems with traditional handlers. This is why you should almost never use signal handlers for IPC, because they're full of footguns and not actually simple to use. Increasing the number of file descriptors doesn't seem like much of a burden. If your app actually pushes on those limits you need to be messing with ulimits anyway.
- ajross 3y ago
  > You don't need a "full protocol parser" for sockets either. You can send/receive C structs just fine. It's also not hard to write to sockets from the command line. Yeah, that's how you get vulnerabilities. For the same reason that "Thou shalt not write async signal handlers without extreme care", thou must never send data down a socket without a clear specification for communication. It doesn't (and shouldn't) have to be a "complicated" protocol. But it absolutely must be a fully specified one. And signals, as mentioned, don't really have that shortcoming due to their extreme simplicity. They're just interrupts and don't pass data (except in the sense that they cause the target to re-inspect state, etc...). They're a tool in the box and you should know how they work. Mostly the point is just "IPC is hard", and choosing tools isn't really the hard part. Signals have their place.

deleted 3y ago

[deleted]

goodyduru 3y ago

Author here. I agree with you on preferring sockets to signals for IPC. It's more straightforward and stable to use it. This article is one of a series I'm writing on IPC. Writing about them ensures I understand them better. I wrote one on sockets[1]. [1] https://goodyduru.github.io/os/2023/10/03/ipc-unix-domain-sockets.html https://goodyduru.github.io/os/2023/10/03/ipc-unix-domain-so...