6 ms·

This appears to be aware that it is solving the same problem as daemontools / runit / nosh / s6... but doesn't seem to offer anything new or better. And it pre

by dsr_ 10y ago

This appears to be aware that it is solving the same problem as daemontools / runit / nosh / s6... but doesn't seem to offer anything new or better. And it presents a solved problem (some daemons fork themselves off and the parent exits, so tell them not to do that or use fghack) as a new thing that Immortal can solve. Is there something actually new and interesting about this that I'm missing?

subway 10y ago
But this one is written in Go!
tyingq 10y ago
There is a "why another supervisor" explanation here: http://immortal.run/about/ http://immortal.run/about/ Mostly, though, covering the point about parent forks and exits that you mention.
- elcritch 10y ago
  Thanks for the project, looks great!
elcritch 10y ago
The "execution" seems better at first blush. The documentation is short and to the point, which means it's something which would lead me to use it, especially since I still haven't figured out how to easily deploy most of the others you mention. The ability to have go produce cross platform binaries would ease a lot of deployment issues as well. Along with distributed logging, it's something I'll probably try using. Unless any of the above have advantages, primarily in the ease of use category.
nbari 10y ago
Instead of using named pipes for managing the process, a Unix socket is been used, so if required this opens the possibility to manage via a web JSON API the services an example about how to use Nginx for this is here https://immortal.run/post/nginx/ https://immortal.run/post/nginx/ The idea behind was to avoid a developer to deal with a DevOps/operator to just stop/restart services, something useful on staging environments. For starting, stopping a process there is no need to unlink/link a file, this simplifies things a little bit, for example, https://immortal.run/post/ansible/ https://immortal.run/post/ansible/ by only updating the file contents or "touching" the file it can be started or re-started. When following a PID, signals can still be sent to the process, these things become handy while working with a lot of microservices. The docs need to be improved, but any feedback, comments or even desired feature is more than welcome.
JdeBP 10y ago
Here are some differences, from a cursory reading: * There is no composable toolset, but rather one monolithic program that does only a small part of the job. envdir and setuidgid are -u and -e options to the supervise analogue that is that monolithic program, "immortal". All of functionality of the other daemontools tools, such as envuidgid and softlimit, let alone the likes of runit's chpst, is missing. Not only are there no composable tools for it, the monolithic program simply lacks their functionality entirely. One can extend this functionality with the composable toolsets, with tools to handle things from scheduling classes to control groups. The monolithic program lacks all of this. * There is no equivalent of the service restart/finalization mechanisms of runit, perp, s6, or nosh; which permit taking various actions dependent from how a service program terminates. * Nor is there an equivalent of the at-stop and at-start mechanisms of s6 or nosh. * There is no service readiness mechanism, as s6 and systemd have. * s6, nosh, and systemd all have the concept of "oneshot" services. immortal does not cater for them. * The svscan analogue, named "immortaldir", lacks an equivalent to the "down" file mechanism from daemontools, runit, perp, s6, et al.. All services always come up at bootstrap. There is no enable/disable mechanism at all. * The author hasn't borne in mind the design lessons from svscan, the reasons why it works as it does, such as only having symbolic links in the scan directory, not the actual service definitions themselves. There is a race condition between copying or saving the YAML file into its directory and immortaldir trying to open and read it, which can result in immortal reading a part-written configuration file. There is also the potential for hidden files to be recognized as services, something that svscan explicitly prohibits. * The author hasn't borne in mind the design lessons from s6-svscan. immortaldir wakes up and polls every few seconds instead of being event-driven, which as M. Bercot notes is a consideration on systems that want to save power. * immortal does not have logger services, as perp, s6, daemontools, daemontools-encore, freedt, nosh, and runit all do. Instead, it has only the options of the service manager itself writing the log file, or a non-supervised non-manageable logging command. It is thus not possible to control logging through service management. * There is no equivalent to multilog's rotation scripts, filtering, multiple log directories, or automatic timestamping. * The log rotation uses the old .1 .2 system. The author hasn't borne in mind the design lessons from multilog, the reasons why it did the things that it did. So it doesn't ensure that logs are safely written to disc at rotation time, for starters. * The service management API is not compatible with daemontools, daemontools-encore, runit, and nosh; which all share a common binary-compatible subset (the daemontools service management API). This API is, rather, a WWW server. Where daemontools et al. send simple one-byte commands; this API uses JSON and the Internet Message format meaning that a simple command to send a signal is actually an entire HTTP/TCP transaction, with all of that parsing and marshalling going on, under the covers. This flies in the face of the "do not parse at runtime" dictum followed by the other toolsets. There is an entire HTTP parser and server running with superuser credentials inside the service manager here. Ironically, this is a criticism that people vehemently (but erroneously) level at systemd. This service manager actually does do this, though. * It also contains an entire YAML parser. * There is no service interdependency mechanism; at all. There aren't even the tools for the old "thundering herd" approach of starting all services and having them manually check their dependencies in their "run" scripts. There is nothing there akin to s6-rc or to the service bundle dependency mechanism in nosh. * There's no mention of portability to the Hurd, Illumos, OpenIndiana, the Windows NT POSIX subsystem, the Windows NT Linux subsystem, and suchlike. Several of the other systems address the fact that "portable" to any "UNIX-like platform" means more than merely FreeBSD, macOS, OpenBSD, and Linux. s6 has notes on how to build it on Solaris, for example. * The new and interesting thing is not the fact that it focusses on the solved problems of fghack and PID files, which ironically have been going away over the past decade. The new and interesting thing is spinning up service and supervise directories on demand from a YAML definition. nosh has the ability to spin up a service directory from a systemd service/socket unit file definition; but the other toolsets leave actually constructing the service and supervise directories from scratch to third parties. That said, such directories are simple enough to construct by hand directly, on the same order of simplicity as the YAML definitions here, in fact. It is notable that the YAML definitions again seem to have features weighted towards PID files and suchlike which are problems that are in the process of disappearing. * One of the fundamental design concepts of the daemontools family is that the service manager knows the process ID of the daemon because it remembers it from when it spawned it and knows that it does not go away until wait(). immortal employs the old broken idea, that this replaced, instead. The code reads process IDs from PID files and tests whether they are valid by sending them signal number 0, in the race-prone and risky way that the world has known not to do for decades. Witness https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64e6b86bbaae9729413e3485/daemon.go#L78 https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64... and https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64e6b86bbaae9729413e3485/daemon.go#L85 https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64... . There is a lot of prior art here that this design is just ignoring. Overall, it is a retrograde step, to a system that is less capable than runit or daemontools are, is less robust, and is less well designed. A better approach is, if one wants to drive services from a WWW browser and to create service definitions from a configuration file, to layer a WWW server and a service definition creation tool over something like s6, which is more complete and more widely portable than immortal. I also recommend that the author read at minimum Daniel J. Bernstein's design doco for daemontools and Laurent Bercot's design doco for s6. M. Bercot goes into some depth when it comes to the reasons why things are engineered as they are. Additionally, the first chapter in the nosh Guide enumerates some of the design fundamentals that underpin it, which daemontools-family systems generally share.

by dsr_ 10y ago

subway 10y ago

But this one is written in Go!

tyingq 10y ago

There is a "why another supervisor" explanation here: http://immortal.run/about/ http://immortal.run/about/ Mostly, though, covering the point about parent forks and exits that you mention.

elcritch 10y ago
Thanks for the project, looks great!

elcritch 10y ago

The "execution" seems better at first blush. The documentation is short and to the point, which means it's something which would lead me to use it, especially since I still haven't figured out how to easily deploy most of the others you mention. The ability to have go produce cross platform binaries would ease a lot of deployment issues as well. Along with distributed logging, it's something I'll probably try using. Unless any of the above have advantages, primarily in the ease of use category.

nbari 10y ago

Instead of using named pipes for managing the process, a Unix socket is been used, so if required this opens the possibility to manage via a web JSON API the services an example about how to use Nginx for this is here https://immortal.run/post/nginx/ https://immortal.run/post/nginx/ The idea behind was to avoid a developer to deal with a DevOps/operator to just stop/restart services, something useful on staging environments. For starting, stopping a process there is no need to unlink/link a file, this simplifies things a little bit, for example, https://immortal.run/post/ansible/ https://immortal.run/post/ansible/ by only updating the file contents or "touching" the file it can be started or re-started. When following a PID, signals can still be sent to the process, these things become handy while working with a lot of microservices. The docs need to be improved, but any feedback, comments or even desired feature is more than welcome.

JdeBP 10y ago

Here are some differences, from a cursory reading: * There is no composable toolset, but rather one monolithic program that does only a small part of the job. envdir and setuidgid are -u and -e options to the supervise analogue that is that monolithic program, "immortal". All of functionality of the other daemontools tools, such as envuidgid and softlimit, let alone the likes of runit's chpst, is missing. Not only are there no composable tools for it, the monolithic program simply lacks their functionality entirely. One can extend this functionality with the composable toolsets, with tools to handle things from scheduling classes to control groups. The monolithic program lacks all of this. * There is no equivalent of the service restart/finalization mechanisms of runit, perp, s6, or nosh; which permit taking various actions dependent from how a service program terminates. * Nor is there an equivalent of the at-stop and at-start mechanisms of s6 or nosh. * There is no service readiness mechanism, as s6 and systemd have. * s6, nosh, and systemd all have the concept of "oneshot" services. immortal does not cater for them. * The svscan analogue, named "immortaldir", lacks an equivalent to the "down" file mechanism from daemontools, runit, perp, s6, et al.. All services always come up at bootstrap. There is no enable/disable mechanism at all. * The author hasn't borne in mind the design lessons from svscan, the reasons why it works as it does, such as only having symbolic links in the scan directory, not the actual service definitions themselves. There is a race condition between copying or saving the YAML file into its directory and immortaldir trying to open and read it, which can result in immortal reading a part-written configuration file. There is also the potential for hidden files to be recognized as services, something that svscan explicitly prohibits. * The author hasn't borne in mind the design lessons from s6-svscan. immortaldir wakes up and polls every few seconds instead of being event-driven, which as M. Bercot notes is a consideration on systems that want to save power. * immortal does not have logger services, as perp, s6, daemontools, daemontools-encore, freedt, nosh, and runit all do. Instead, it has only the options of the service manager itself writing the log file, or a non-supervised non-manageable logging command. It is thus not possible to control logging through service management. * There is no equivalent to multilog's rotation scripts, filtering, multiple log directories, or automatic timestamping. * The log rotation uses the old .1 .2 system. The author hasn't borne in mind the design lessons from multilog, the reasons why it did the things that it did. So it doesn't ensure that logs are safely written to disc at rotation time, for starters. * The service management API is not compatible with daemontools, daemontools-encore, runit, and nosh; which all share a common binary-compatible subset (the daemontools service management API). This API is, rather, a WWW server. Where daemontools et al. send simple one-byte commands; this API uses JSON and the Internet Message format meaning that a simple command to send a signal is actually an entire HTTP/TCP transaction, with all of that parsing and marshalling going on, under the covers. This flies in the face of the "do not parse at runtime" dictum followed by the other toolsets. There is an entire HTTP parser and server running with superuser credentials inside the service manager here. Ironically, this is a criticism that people vehemently (but erroneously) level at systemd. This service manager actually does do this, though. * It also contains an entire YAML parser. * There is no service interdependency mechanism; at all. There aren't even the tools for the old "thundering herd" approach of starting all services and having them manually check their dependencies in their "run" scripts. There is nothing there akin to s6-rc or to the service bundle dependency mechanism in nosh. * There's no mention of portability to the Hurd, Illumos, OpenIndiana, the Windows NT POSIX subsystem, the Windows NT Linux subsystem, and suchlike. Several of the other systems address the fact that "portable" to any "UNIX-like platform" means more than merely FreeBSD, macOS, OpenBSD, and Linux. s6 has notes on how to build it on Solaris, for example. * The new and interesting thing is not the fact that it focusses on the solved problems of fghack and PID files, which ironically have been going away over the past decade. The new and interesting thing is spinning up service and supervise directories on demand from a YAML definition. nosh has the ability to spin up a service directory from a systemd service/socket unit file definition; but the other toolsets leave actually constructing the service and supervise directories from scratch to third parties. That said, such directories are simple enough to construct by hand directly, on the same order of simplicity as the YAML definitions here, in fact. It is notable that the YAML definitions again seem to have features weighted towards PID files and suchlike which are problems that are in the process of disappearing. * One of the fundamental design concepts of the daemontools family is that the service manager knows the process ID of the daemon because it remembers it from when it spawned it and knows that it does not go away until wait(). immortal employs the old broken idea, that this replaced, instead. The code reads process IDs from PID files and tests whether they are valid by sending them signal number 0, in the race-prone and risky way that the world has known not to do for decades. Witness https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64e6b86bbaae9729413e3485/daemon.go#L78 https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64... and https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64e6b86bbaae9729413e3485/daemon.go#L85 https://github.com/immortal/immortal/blob/9ed9087a8ef1e64d64... . There is a lot of prior art here that this design is just ignoring. Overall, it is a retrograde step, to a system that is less capable than runit or daemontools are, is less robust, and is less well designed. A better approach is, if one wants to drive services from a WWW browser and to create service definitions from a configuration file, to layer a WWW server and a service definition creation tool over something like s6, which is more complete and more widely portable than immortal. I also recommend that the author read at minimum Daniel J. Bernstein's design doco for daemontools and Laurent Bercot's design doco for s6. M. Bercot goes into some depth when it comes to the reasons why things are engineered as they are. Additionally, the first chapter in the nosh Guide enumerates some of the design fundamentals that underpin it, which daemontools-family systems generally share.