4 ms·
Why would you use pid files when you've got systemd available? All the process management can be done directly with children or cgroups for some crazy runaway p
by viraptor 10y ago
Why would you use pid files when you've got systemd available? All the process management can be done directly with children or cgroups for some crazy runaway processes.
- JdeBP 10y agoOr, more generally: > Why would you use PID files when you've got proper service management available? as of course the flawed PID file mechanism was addressed long before systemd was even an idea. cgroups do not solve the problem of "crazy runaway processes", by the way. cgroups are not jobs (in the Windows NT and VMS senses), and lack the functionality of a true job mechanism. When systemd terminates all of the processes in a cgroup, it doesn't issue a single "terminate job" system call. There isn't such a thing. It sits in a loop in application-mode code repeatedly scanning all of the process IDs in the cgroup (by re-reading a file full of PID numbers) and sending signals to new processes that it hasn't seen before. There are several problems with this. A first problem is that systemd can be slower than whatever is reaping child processes within the process group, leading to the termination signals being sent to completely the wrong process: one that just happened to re-use the same process ID in between systemd reading the cgroup's process list file and it actually getting around to sending signals to the list of processes. The order of events is: (1) systemd reads PID N from the cgroup's process ID list, then its timeslice happens to expire; (2) process N is scheduled and exits; (3) process N's reaper (its parent, or a sub-reaper) cleans up the zombie in the process table; (4) some vital part of the system is scheduled and spawns a new process re-using ID N; (5) systemd is finally scheduled, works down the list of PIDs that it read before, and merrily kills the new process. A second problem is that a program that forks new processes quickly enough within the cgroup can keep systemd spinning for a long time, in theory indefinitely as long as suitable "weather" prevails, as at each loop iteration there will be one more process to kill. Note that this does not have to be a fork bomb. It only has to fork enough that systemd sees at least one more new process ID in the cgroup every time that it runs its loop. A third problem is that systemd keeps process IDs of processes that it has already signalled in a set, to know which ones it won't try to send signals to again. It's possible that a process with ID N could be signalled, terminate, and be cleaned out of the process table by a reaper/parent; and then something within the cgroup fork a new process that is allocated the same process ID N once again. systemd will re-read the cgroup's process ID list, think that it has already signalled the new process, and not signal it at all. These are addressed by a true "job" mechanism. But cgroups are not such.
- viraptor 10y agoSure, it's still not perfect. What I meant with cgroups for runaway processes is that at least you still see which group they belong to. We've still got to solve the "kill the whole cgroup" idea.
- TheDong 10y agoYou can also just freeze the whole cgroup, and then the kernel will cascade it to all tasks within said cgroup, and then you can do process management. Most of what you complain about though is solved by duplicate pids not being assigned in short succession (which is generally true, though in reality is an implementation detail of linux which is not promised).
- JdeBP 10y agoI was expecting someone to mention the freezer. Not only does systemd not use the freezer, but the systemd people explicitly describe it as having "broken inheritance semantics of the kernel logic". You'll have to ask them what they mean by that, but the freezer does not, for them, magically turn cgroups into a job mechanism either. Which leads to another point. If you think that this is my complaint, you should read the systemd people on the subject over the years. They were rather expecting (version 2) cgroups to be a true job mechanism, and they've complained every time that it has once again turned out not to be. Moreover: this is not to mention that Docker and others will manipulate the freeze status of control groups for their own purposes, and there is no real race-free mechanism for sharing this setting amongst multiple owners, such as an atomic read-and-update for it.
- wahern 10y agosystemd DOES NOT solve the PID file race. What is does is poll a cgroups proc file that lists the PIDs in a cgroup, then iteratively calls kill on those PIDs. But this is _just_ as racy as a regular PID file. I explain more here, with links to the relevant bits of code: https://lwn.net/Articles/677671/ Long story short, Linux does not offer a way to atomically send a signal to a cgroup, and so there's no way for systemd to reliably kill errant (or worse, malicious) processes that keep forking. Worse, last time I checked systemd had a bug in their read-iterate-kill loop that could lead to an infinite loop. I also detail that issue in the above LWN thread. If you _carefully_ parse Pottering's statements regarding systemd, a charitable interpretation would be that he never claimed systemd solved the PID race for uncooperative software. Unfortunately, everybody believed and most people still believe that it does. Afterall, there was never a problem with PID races with _cooperative_ software, so it was natural to assume that what systemd was bringing to the table was a solution to uncooperative processes. That doesn't mean systemd isn't slightly better then `killall [process name pattern]`. But in security an inherently broken solution isn't any solution at all.
- viraptor 10y ago> systemd DOES NOT solve the PID file race I did not claim otherwise. And I wouldn't even describe it as PID file race. PID files are a specific thing that was used with older inits. I don't believe any init can solve the issue you're describing - it exists on a different layer. But cgroups freezer should be able to solve it.
- wahern 10y agoI thought your question implied that systemd was superior to PID files because it solved the PID race. To answer your question more directly, why do I use PID files: 1) Because I write and value portable software, both across different Linux distros and across different POSIX-like systems. I regularly target AIX, FreeBSD, NetBSD, OpenBSD, OS X, and Solaris; none as second class citizens. 2) Because IME the compile-test-debug cycle and similar non-production runtime modes is easier when you don't have to fiddle with separate init files. Supporting a simple PID-file-based backgrounding capability directly in your software is a relatively simple matter when not implemented as an after-thought. 3) Because sometimes you want to implement a service using multiple processes (e.g. worker pools, specialized privilege separation, etc), which is easier (both in development and production) when you fork them from your own master rather than trying to configure and coordinate through a third-party supervisor. Note that this doesn't exclude your service's supervisor running in the foreground under another supervisor like init (e.g. systemd) or daemontools. 4) Because using process groups and a shared controlling pseudo-terminal you _can_ atomically send a signal to all children in a process group. It's more robust than what systemd currently does with cgroups, it's portable as a practical matter, and relatively simple to accomplish early in main(). Note that I write my software to support running in the foreground under the control of any supervisor of choice, including support for inheriting a listening socket. This is always the default mode. Backgrounding, PID files, etc, are optional capabilities in my daemons. Regarding cgroups freezer, it does seem like it should work. I stand corrected wrt Linux supporting atomic signals to an entire cgroup. But it seems like neither systemd nor Docker has actually implemented support. Perhaps it's for lack of motivation, or perhaps because there are non-trivial issues to resolve. It would be interesting to know what the hold up is.