10 ms·
How Supervisors Work
- rdtsc 10y agoI also like this resource on supervisors: http://learnyousomeerlang.com/supervisors#from-bad-to-good http://learnyousomeerlang.com/supervisors#from-bad-to-good This is another terse (more reference-like) description of strategies: http://erlang.org/doc/design_principles/sup_princ.html#strategy http://erlang.org/doc/design_principles/sup_princ.html#strat...
- dajohnson89 10y agoObligatory joke: supervisors don't work, they just micromanage others :)
- zerd 10y agoI'M the Supervisor. Can I get a taxi number? https://en.wikipedia.org/wiki/IM_the_Supervisor https://en.wikipedia.org/wiki/IM_the_Supervisor
- andrewchambers 10y agoIs anyone aware of a supervisor system like otp supervisors that manage unix processes? I'm trying to throw together a collection of small services on a server in a reliable way, and there doesn't seem to be anything that does this on the OS level.
- Ixiaus 10y agoA few ideas come to mind: 1) Use systemd; I've taken the time to learn it and the tooling it exposes for codifying process dependencies, how weak or strong they should be, and restart behaviors is thoroughly designed 2) Use Erlang/OTP - I went this route for an IoT product I worked on that used Erlang as the primary high-level "firmware", I codified the process tree and dependencies and custom supervisor restart behaviors for unix processes into Erlang/OTP's supervision tree; using Erlang process ports The second approach has worked extremely well for me in the past but I would only recommend it if you're going to use a lot of Erlang to process output from the programs it spawns for you, which was the case for how I was using it at the time, the robust supervision of the processes it managed in a supervision tree was incidental to the primary use but extremely robust.
- tonyarkles 10y agoAny chance the 2nd one was Nerves-based?
- deleted 10y ago[deleted]
- Ixiaus 10y agoNo it was not, it was a home-rolled solution. Nerves is pretty neat but at the time I wrote our solution it wasn't as mature as it is these days. I rolled our own buildroot + erlang release + "firmware" deployment system (including the cloud infrastructure and software). Very fun project.
- tonyarkles 10y agoVery cool! I think the platform has a pile of potential for small devices, I just haven't had a project yet where I feel like I could justify the learning curve. Getting BEAM running on a beaglebone via Nerves was pretty neat, but led to a bit of a "ok cool, now what?" situation.
- andrewchambers 10y agoWriting a unix process supervisor system in erlang itself as a thin wrapper over OTP might be a fun project. Cool idea!
- pmontra 10y agoSupervisord, upstart, systemd, monit and a bunch of other systems seem the obvious answers, so obvious that I probably misunderstood the question. Maybe you're looking for supervision trees or "start this after you started that"?
- andrewchambers 10y agoDo any of those tools to things like kill sibling processes to ensure a stable restart point? or allow deeply nested process trees with faults propagating upward if restarts aren't working?
- scrollaway 10y agoQuestion: I'm currently using supervisor on systemd systems to manage uwsgi type processes. Am I doing it wrong? Should I be creating systemd units?
- JdeBP 10y agoHere's the doco, showing the creation of systemd service units. * http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
- JdeBP 10y agoThere are plenty of things doing this at the operating system level, going back a quarter of a century. One of the earliest was the System Resource Controller in IBM AIX 3.2 in 1992. Later that same decade, in the BSD and Linux worlds, came Daniel J. Bernstein's daemontools. A whole family of toolsets grew from that design, including Bruce Guenter's daemontools-encore, Gerrit Pape's runit, Laurent Bercot's s6, Wayne Marshall's perp, Adam Sampson's freedt, and nosh. There are also upstart, Linux systemd, MacOS launchd, and the Solaris Service Management Facility. * http://jdebp.eu./FGA/daemontools-family.html http://jdebp.eu./FGA/daemontools-family.html
- andrewchambers 10y agoDaemon tools most certainly does not do what im talking about. It does blind restarts of single processes, not coordinated restarts and killing of collections of services.
- JdeBP 10y agoYou are voicing a long-standing fallacy. One builds the coördination on top of the base service management. The runit and perp way is to include the dependencies as calls to "sv check"/"perpok" in the run programs. s6-rc gives one "bundles". nosh has a mechanism of targets and "wants"/"conflicts" relationships.
- Ixiaus 10y agoSupervisors aren't just for restarting processes that crash, they're also responsible for cleaning up processes when the tree is being shutdown. If you're using spawn to fire off a long-ish running process that is too small in scope to justify making a gen_<something>, you might be doing it wrong. You can easily affix the logic you need as a process on a supervisor which will (very importantly) help clean up those processes for you. A common pattern is to use a high-level supervisor for starting up the primary gen_<whatever>'s with sub-supervisors responsible for short-lived and long-lived "small logic" children - you can also ask a supervisor to add and start a new child and to remove one, let the supervisors do the bookkeeping! Supervisors make your process trees declarative and help you avoid process soup, even when the processes aren't that important. I highly recommend harnessing that as much as possible when designing non-trivial Erlang programs.
- dvcrn 10y agoInteresting thoughts, thank you! I myself am guilty of spawning a few processes here and there that I consider not important enough to put into a supervisor.
- greenleafjacob 10y agoI think the fact that Erlang provides an incentive for declaring what processes are managed upfront is a great benefit. I find myself planning a bit the structure of the supervision tree whereas in Go there is no benefit so it's much more ad hoc and messy.
- jerf 10y agoI wrote suture for that: https://github.com/thejerf/suture https://github.com/thejerf/suture Do check out the "blog post describing the design decisions" I mention in the README.md; you don't need to tell me every precise way in which it differs from Erlang and why Go can't be exactly like Erlang, I've already done that for you. Like you, I have found it useful for structuring programs in many ways moreso than for reliability. That said, the reliability does sometimes end up being helpful.
- pmontra 10y agoGreat digging into the Elixir and Erlang code. By the way, am I the only one feeling that having to write the source code of the supervisor of a process is little more than boilerplate and the language should offer that by default as part? Obviously we want a supervisor to supervise the process and add it to our supervision tree. It could be an argument of spawn, something like spawn :supervised, -> function end (maybe not the best syntax, it's only to show the idea). That would be a kind of anonymous supervisor. We should write our own supervisor code only to handle special cases.
- di4na 10y agoyou could, that is what Task.Supervisor do. But you then run into other problems. Who is supervising that supervisor? What are the strategy? What are the name? In which order to clean the process on shutdown ? When in the init should we spawn it? In the end, it goes down to making a high level dynamic supervisor and putting everything under it. Which... is quite the opposite goal of a supervisor.
- pmontra 10y agoMy idea is to provide sensible defaults and write custom code to handle everything else. What we do now is to write custom code every time. My point is that the custom code we write is almost always the same for every process in the application. It hurts productivity. Let's use your list of questions as a checklist to validate the feasibility of this approach: > Who is supervising that supervisor? The supervisor of the process that spawned that process. Would that always work? Or the top supervisor of the application. Or your "high level dynamic supervisor" in the language runtime itself, which supervises only those anonymous supervisors and not every process. > What are the strategy? one-for-one should be a good default. > What are the name? Something automatically generated. A uuid. I'm not caring about that supervisor anyway. Would that work? > In which order to clean the process on shutdown? If I'm not caring about that process enough to write its supervisor explicitly maybe I don't care about its turn in the shutdown order. Anything easy to implement will do. > When in the init should we spawn it? Never, it's created only when the developer explicitly spawns the process. Well, we could reach the point when mix has a default task which spawns some processes with anonymous supervisors and there are no explicit supervisors left. This could change a little the nature of the language. I elaborate on this: Erlang/Elixir processes are already somewhat equivalent to objects: they communicate with message passing, conceptually similar to Smalltalk and many object oriented languages (Ruby has send, JS has apply, etc). The main difference is that Erlang processes have their own CPU and are naturally concurrent, which is great. If supervision can be made so easy to disappear inside the language, perhaps processes become even more conceptually like objects, created when needed (a "spawn" is a "new") instead of carefully planned entities much like system administrators plan services in upstart/systemd. This could be good or bad, I don't know, but that every object has its CPU feels right. BTW, google for Joe Armstrong Erlang is the only true object oriented language. It's not heresy :-)
- throwbsidbdk 10y agoIs it not cool to call these things worker threads anymore? Nothing wrong with the article but I'm getting annoyed by the trend to rename constructs and common patterns in new languages. Wasting time learning about Y in language B only to realize that it's exactly the same as Z in language A which you already knew
- lightbyte 10y agoThese constructs were designed for erlang as far back at 1986. They also are not just worker threads, they do not have a similar concept in most languages.
- ramchip 10y agoErlang processes are quite different from worker threads in other languages, since they're green threads (not OS threads). I'm not even sure if the expression "worker thread" was in common usage when the language was created.
- throwbsidbdk 10y agoI didn't know Erlang had been around that long, I've only seen it pop up within the last few years
- lightbyte 10y agoI think you meant to reply that to me. But yeah, erlang is very old. It was originally designed from the ground up to run telephony switches and other telecom-related tasks.