6 ms·

Is anyone aware of a supervisor system like otp supervisors that manage unix processes? I'm trying to throw together a collection of small services on a server

Is anyone aware of a supervisor system like otp supervisors that manage unix processes? I'm trying to throw together a collection of small services on a server in a reliable way, and there doesn't seem to be anything that does this on the OS level.

Ixiaus 10y ago
A few ideas come to mind: 1) Use systemd; I've taken the time to learn it and the tooling it exposes for codifying process dependencies, how weak or strong they should be, and restart behaviors is thoroughly designed 2) Use Erlang/OTP - I went this route for an IoT product I worked on that used Erlang as the primary high-level "firmware", I codified the process tree and dependencies and custom supervisor restart behaviors for unix processes into Erlang/OTP's supervision tree; using Erlang process ports The second approach has worked extremely well for me in the past but I would only recommend it if you're going to use a lot of Erlang to process output from the programs it spawns for you, which was the case for how I was using it at the time, the robust supervision of the processes it managed in a supervision tree was incidental to the primary use but extremely robust.
- tonyarkles 10y ago
  Any chance the 2nd one was Nerves-based?
  - deleted 10y ago
    [deleted]
  - Ixiaus 10y ago
    No it was not, it was a home-rolled solution. Nerves is pretty neat but at the time I wrote our solution it wasn't as mature as it is these days. I rolled our own buildroot + erlang release + "firmware" deployment system (including the cloud infrastructure and software). Very fun project.
    - tonyarkles 10y ago
      Very cool! I think the platform has a pile of potential for small devices, I just haven't had a project yet where I feel like I could justify the learning curve. Getting BEAM running on a beaglebone via Nerves was pretty neat, but led to a bit of a "ok cool, now what?" situation.
- andrewchambers 10y ago
  Writing a unix process supervisor system in erlang itself as a thin wrapper over OTP might be a fun project. Cool idea!
pmontra 10y ago
Supervisord, upstart, systemd, monit and a bunch of other systems seem the obvious answers, so obvious that I probably misunderstood the question. Maybe you're looking for supervision trees or "start this after you started that"?
- andrewchambers 10y ago
  Do any of those tools to things like kill sibling processes to ensure a stable restart point? or allow deeply nested process trees with faults propagating upward if restarts aren't working?
- scrollaway 10y ago
  Question: I'm currently using supervisor on systemd systems to manage uwsgi type processes. Am I doing it wrong? Should I be creating systemd units?
  - JdeBP 10y ago
    Here's the doco, showing the creation of systemd service units. * http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
JdeBP 10y ago
There are plenty of things doing this at the operating system level, going back a quarter of a century. One of the earliest was the System Resource Controller in IBM AIX 3.2 in 1992. Later that same decade, in the BSD and Linux worlds, came Daniel J. Bernstein's daemontools. A whole family of toolsets grew from that design, including Bruce Guenter's daemontools-encore, Gerrit Pape's runit, Laurent Bercot's s6, Wayne Marshall's perp, Adam Sampson's freedt, and nosh. There are also upstart, Linux systemd, MacOS launchd, and the Solaris Service Management Facility. * http://jdebp.eu./FGA/daemontools-family.html http://jdebp.eu./FGA/daemontools-family.html
- andrewchambers 10y ago
  Daemon tools most certainly does not do what im talking about. It does blind restarts of single processes, not coordinated restarts and killing of collections of services.
  - JdeBP 10y ago
    You are voicing a long-standing fallacy. One builds the coördination on top of the base service management. The runit and perp way is to include the dependencies as calls to "sv check"/"perpok" in the run programs. s6-rc gives one "bundles". nosh has a mechanism of targets and "wants"/"conflicts" relationships.

Ixiaus 10y ago

A few ideas come to mind: 1) Use systemd; I've taken the time to learn it and the tooling it exposes for codifying process dependencies, how weak or strong they should be, and restart behaviors is thoroughly designed 2) Use Erlang/OTP - I went this route for an IoT product I worked on that used Erlang as the primary high-level "firmware", I codified the process tree and dependencies and custom supervisor restart behaviors for unix processes into Erlang/OTP's supervision tree; using Erlang process ports The second approach has worked extremely well for me in the past but I would only recommend it if you're going to use a lot of Erlang to process output from the programs it spawns for you, which was the case for how I was using it at the time, the robust supervision of the processes it managed in a supervision tree was incidental to the primary use but extremely robust.

tonyarkles 10y ago
Any chance the 2nd one was Nerves-based?
- deleted 10y ago
  [deleted]
- Ixiaus 10y ago
  No it was not, it was a home-rolled solution. Nerves is pretty neat but at the time I wrote our solution it wasn't as mature as it is these days. I rolled our own buildroot + erlang release + "firmware" deployment system (including the cloud infrastructure and software). Very fun project.
  - tonyarkles 10y ago
    Very cool! I think the platform has a pile of potential for small devices, I just haven't had a project yet where I feel like I could justify the learning curve. Getting BEAM running on a beaglebone via Nerves was pretty neat, but led to a bit of a "ok cool, now what?" situation.
andrewchambers 10y ago
Writing a unix process supervisor system in erlang itself as a thin wrapper over OTP might be a fun project. Cool idea!

pmontra 10y ago

Supervisord, upstart, systemd, monit and a bunch of other systems seem the obvious answers, so obvious that I probably misunderstood the question. Maybe you're looking for supervision trees or "start this after you started that"?

andrewchambers 10y ago
Do any of those tools to things like kill sibling processes to ensure a stable restart point? or allow deeply nested process trees with faults propagating upward if restarts aren't working?
scrollaway 10y ago
Question: I'm currently using supervisor on systemd systems to manage uwsgi type processes. Am I doing it wrong? Should I be creating systemd units?
- JdeBP 10y ago
  Here's the doco, showing the creation of systemd service units. * http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html

JdeBP 10y ago

There are plenty of things doing this at the operating system level, going back a quarter of a century. One of the earliest was the System Resource Controller in IBM AIX 3.2 in 1992. Later that same decade, in the BSD and Linux worlds, came Daniel J. Bernstein's daemontools. A whole family of toolsets grew from that design, including Bruce Guenter's daemontools-encore, Gerrit Pape's runit, Laurent Bercot's s6, Wayne Marshall's perp, Adam Sampson's freedt, and nosh. There are also upstart, Linux systemd, MacOS launchd, and the Solaris Service Management Facility. * http://jdebp.eu./FGA/daemontools-family.html http://jdebp.eu./FGA/daemontools-family.html

andrewchambers 10y ago
Daemon tools most certainly does not do what im talking about. It does blind restarts of single processes, not coordinated restarts and killing of collections of services.
- JdeBP 10y ago
  You are voicing a long-standing fallacy. One builds the coördination on top of the base service management. The runit and perp way is to include the dependencies as calls to "sv check"/"perpok" in the run programs. s6-rc gives one "bundles". nosh has a mechanism of targets and "wants"/"conflicts" relationships.