7 ms·

We still run daemontools even in systemd systems. The OS packaged cruft runs in systemd. Everything important runs in daemontools. Still the best "services mana

by scurvy 11y ago

We still run daemontools even in systemd systems. The OS packaged cruft runs in systemd. Everything important runs in daemontools. Still the best "services manager" out there.

otterley 11y ago
Why? It's no longer necessary. Also, daemontools suffers from the same problem as other non-systemd service managers; it can't manage processes that daemonize themselves for good reason (like nginx or Unicorn). systemd can handle it because it creates a unique cgroup for every service; if the service needs to be stopped, all processes in the cgroup are terminated.
- vezzy-fnord 11y ago
  There's plenty of successors to daemontools following the same principles, but more advanced: s6, nosh, perp, runit, etc. That said, from what I know systemd doesn't necessarily handle double-forking daemons any better. That is to say, if you set the service to Type=forking, it'll still need to use a PID file (with its inherent race conditions) or employ a PID guessing heuristic, which can fail. This is a universal problem that no hack can truly solve. The bottom line is if you're running under a service manager, you don't daemonize. You delegate to the service manager to daemonize for you. Any deviation from this will be finicky on most Unix-likes. The purpose of cgroups here is instead process tracking, i.e. reliably killing all children. But for some services this is exactly what you don't want, and then there's nothing stopping you from allocating cgroups yourself if needed, or talk to a cgroup hierarchy management daemon like cgmanager. Ultimately, you just want some unit of isolation here, so you can use whatever your platform has, i.e. jails or contracts.
  - thwarted 11y ago
    One little known feature added in Linux 3.4 is the PR_SET_CHILD_SUBREAPER prctl(2) [0] that lets you designate the process as being a "subreaper" and it will be notified of any descending subprocess exits, just like init would be. [0] http://man7.org/linux/man-pages/man2/prctl.2.html http://man7.org/linux/man-pages/man2/prctl.2.html
    - vezzy-fnord 11y ago
      Yes, that was actually proposed by Poettering himself. DragonFly BSD has a similar mechanism with procctl(2).
      - thwarted 11y ago
        Yeah, I did research a while ago to find out where that patch came from when there was a lot of noise about how systemd needed to be pid 1.
  - digi_owl 11y ago
    As a bit of an aside, cgmanager got kinda sidelined thanks to the cgroups kernel maintainer at one point insisting on there being only one cgroup management process. And as systemd did both init and cgroup management, it ended up being favored...
- scurvy 11y ago
  We run both nginx and unicorn through daemontools. Both have options to prevent daemonization. Why run something terrible (systemd) when we already have something that works really well (daemontools)?
  - KirinDave 11y ago
    > Why run something terrible (systemd) It is terrible only because you say it is. It's been a massive help for us in adopting it, and accepting the help it offers makes custom software systems a lot easier to write in a robust way. It's only "terrible" if you have axe to grind about the "right" way and then define that as daemontools.
    - cgag 11y ago
      I use systemd to start runit which is like daemontools. I consider daemontools/runit it right because they gives me all the benefits I'd want out of systemd (not doing anything special to daemonize my process, supervision, log handling) while being much simpler and more well tested.
      - KirinDave 11y ago
        > while being much simpler The sum of the init functional that can deliver what systemd is doing (and it is good to do those things) is more complex than a suite of tools for launching daemons in an unhelpful environment. Sure. I submit the universal configuration file tooling is fantastic in a day and age where we provision and configure machines with template-based tools like salt and ansible rather than synchronized command tools like puppet.
        cgag 11y ago
        I'm not sure I understand what you're saying. Is it that it's nice to be able to copy over a systemd service template file with ansible vs running commands to start services? If so, how is that different from copying runit service files instead? If not, can you elaborate / reword?
- pmahoney 11y ago
  > unique cgroup for every service This is a nice feature of systemd. I've been using s6 quite a bit (of daemontools heritage), and often want such a feature, but part of the reason I'm using s6 is so that I can use the same run scripts in darwin and Linux, and only Linux has cgroups.
- pmahoney 11y ago
  > processes that daemonize themselves for good reason (like nginx or Unicorn) Is there a good reason? I assume you're thinking of the ability to fork and exec a new master process, running, e.g. a new version of nginx, then handing control to that new master without dropping any connections, possibly even waiting around a bit in case the new master dies and the original master needs to take over again. I suspect with something like s6-fdholder-daemon [1] could be used to orchestrate a similar process, though I'm not enough of an expert to know. Instead of inheriting file descriptors through forking, an entirely separate, supervised nginx-newversion could be started, get the file descriptors from fdholder, then coordinate with nginx-oldversion about who's going to accept new connections. Certainly depending on such a service is arguably more complex then just having the functionality built-in. On the other hand, just as every service should not implement its own daemonization, one can argue each service should not implement its own hot restarting. If the technique was more common, the argument to not duplicate such code would be stronger. [1] http://www.skarnet.org/software/s6/s6-fdholder-daemon.html http://www.skarnet.org/software/s6/s6-fdholder-daemon.html
  - daurnimator 11y ago
    > I suspect with something like s6-fdholder-daemon [1] could be used to orchestrate a similar process, systemd has the ability to hold fds for you. You need to enable it for each service with http://www.freedesktop.org/software/systemd/man/systemd.service.html#FileDescriptorStoreMax= http://www.freedesktop.org/software/systemd/man/systemd.serv... and the service needs to know hot to actually do it.
  - skarnet 11y ago
    Yes, that was exactly the idea behind s6-fdholder-daemon: set up a central server to keep fds open when you need to restart a process. The old process stores the fd into the fdholder, then dies; the supervisor starts the new process, that retrieves the fd from the fdholder, and starts serving. And if you don't want to use a supervisor or a fdholder, you don't even need to coordinate, and you never need to fork: simply re-exec your executable with your serving socket in a conventional place (stdin is good). Daemons should be able to take a preopened listening socket and serve on it; hot-restarting is then a simple matter of one execve(). There's really no reason to make it more complicated.
rodgerd 11y ago
It says a lot about HN's general self-styled adherence to meritocracy that a stupid, baroque, and ill-thought out technical decision driven by political spite is top-ranked on a system discussion.
- ossreality 11y ago
  rodgerd is the only one with any sense in this thread.

by scurvy 11y ago

We still run daemontools even in systemd systems. The OS packaged cruft runs in systemd. Everything important runs in daemontools. Still the best "services manager" out there.

otterley 11y ago

Why? It's no longer necessary. Also, daemontools suffers from the same problem as other non-systemd service managers; it can't manage processes that daemonize themselves for good reason (like nginx or Unicorn). systemd can handle it because it creates a unique cgroup for every service; if the service needs to be stopped, all processes in the cgroup are terminated.

vezzy-fnord 11y ago
There's plenty of successors to daemontools following the same principles, but more advanced: s6, nosh, perp, runit, etc. That said, from what I know systemd doesn't necessarily handle double-forking daemons any better. That is to say, if you set the service to Type=forking, it'll still need to use a PID file (with its inherent race conditions) or employ a PID guessing heuristic, which can fail. This is a universal problem that no hack can truly solve. The bottom line is if you're running under a service manager, you don't daemonize. You delegate to the service manager to daemonize for you. Any deviation from this will be finicky on most Unix-likes. The purpose of cgroups here is instead process tracking, i.e. reliably killing all children. But for some services this is exactly what you don't want, and then there's nothing stopping you from allocating cgroups yourself if needed, or talk to a cgroup hierarchy management daemon like cgmanager. Ultimately, you just want some unit of isolation here, so you can use whatever your platform has, i.e. jails or contracts.
- thwarted 11y ago
  One little known feature added in Linux 3.4 is the PR_SET_CHILD_SUBREAPER prctl(2) [0] that lets you designate the process as being a "subreaper" and it will be notified of any descending subprocess exits, just like init would be. [0] http://man7.org/linux/man-pages/man2/prctl.2.html http://man7.org/linux/man-pages/man2/prctl.2.html
  - vezzy-fnord 11y ago
    Yes, that was actually proposed by Poettering himself. DragonFly BSD has a similar mechanism with procctl(2).
    - thwarted 11y ago
      Yeah, I did research a while ago to find out where that patch came from when there was a lot of noise about how systemd needed to be pid 1.
- digi_owl 11y ago
  As a bit of an aside, cgmanager got kinda sidelined thanks to the cgroups kernel maintainer at one point insisting on there being only one cgroup management process. And as systemd did both init and cgroup management, it ended up being favored...
scurvy 11y ago
We run both nginx and unicorn through daemontools. Both have options to prevent daemonization. Why run something terrible (systemd) when we already have something that works really well (daemontools)?
- KirinDave 11y ago
  > Why run something terrible (systemd) It is terrible only because you say it is. It's been a massive help for us in adopting it, and accepting the help it offers makes custom software systems a lot easier to write in a robust way. It's only "terrible" if you have axe to grind about the "right" way and then define that as daemontools.
  - cgag 11y ago
    I use systemd to start runit which is like daemontools. I consider daemontools/runit it right because they gives me all the benefits I'd want out of systemd (not doing anything special to daemonize my process, supervision, log handling) while being much simpler and more well tested.
    - KirinDave 11y ago
      > while being much simpler The sum of the init functional that can deliver what systemd is doing (and it is good to do those things) is more complex than a suite of tools for launching daemons in an unhelpful environment. Sure. I submit the universal configuration file tooling is fantastic in a day and age where we provision and configure machines with template-based tools like salt and ansible rather than synchronized command tools like puppet.
      - cgag 11y ago
        I'm not sure I understand what you're saying. Is it that it's nice to be able to copy over a systemd service template file with ansible vs running commands to start services? If so, how is that different from copying runit service files instead? If not, can you elaborate / reword?
pmahoney 11y ago
> unique cgroup for every service This is a nice feature of systemd. I've been using s6 quite a bit (of daemontools heritage), and often want such a feature, but part of the reason I'm using s6 is so that I can use the same run scripts in darwin and Linux, and only Linux has cgroups.
pmahoney 11y ago
> processes that daemonize themselves for good reason (like nginx or Unicorn) Is there a good reason? I assume you're thinking of the ability to fork and exec a new master process, running, e.g. a new version of nginx, then handing control to that new master without dropping any connections, possibly even waiting around a bit in case the new master dies and the original master needs to take over again. I suspect with something like s6-fdholder-daemon [1] could be used to orchestrate a similar process, though I'm not enough of an expert to know. Instead of inheriting file descriptors through forking, an entirely separate, supervised nginx-newversion could be started, get the file descriptors from fdholder, then coordinate with nginx-oldversion about who's going to accept new connections. Certainly depending on such a service is arguably more complex then just having the functionality built-in. On the other hand, just as every service should not implement its own daemonization, one can argue each service should not implement its own hot restarting. If the technique was more common, the argument to not duplicate such code would be stronger. [1] http://www.skarnet.org/software/s6/s6-fdholder-daemon.html http://www.skarnet.org/software/s6/s6-fdholder-daemon.html
- daurnimator 11y ago
  > I suspect with something like s6-fdholder-daemon [1] could be used to orchestrate a similar process, systemd has the ability to hold fds for you. You need to enable it for each service with http://www.freedesktop.org/software/systemd/man/systemd.service.html#FileDescriptorStoreMax= http://www.freedesktop.org/software/systemd/man/systemd.serv... and the service needs to know hot to actually do it.
- skarnet 11y ago
  Yes, that was exactly the idea behind s6-fdholder-daemon: set up a central server to keep fds open when you need to restart a process. The old process stores the fd into the fdholder, then dies; the supervisor starts the new process, that retrieves the fd from the fdholder, and starts serving. And if you don't want to use a supervisor or a fdholder, you don't even need to coordinate, and you never need to fork: simply re-exec your executable with your serving socket in a conventional place (stdin is good). Daemons should be able to take a preopened listening socket and serve on it; hot-restarting is then a simple matter of one execve(). There's really no reason to make it more complicated.

rodgerd 11y ago

It says a lot about HN's general self-styled adherence to meritocracy that a stupid, baroque, and ill-thought out technical decision driven by political spite is top-ranked on a system discussion.

ossreality 11y ago
rodgerd is the only one with any sense in this thread.