3 ms·

I'm intrigued by the idea of mutual supervision. This implies that both process dynamically become the supervisor of each other. By default, OTP supervisors are

I'm intrigued by the idea of mutual supervision. This implies that both process dynamically become the supervisor of each other. By default, OTP supervisors are more architectural in nature -- they will always give you a top-down tree rather than a cyclic graph. Who restarts who? That's a bit tougher to do when everything becomes cyclic. Links are bidirectional though, so when one of the workers die, the other can know about it, die, propagate the error or react to it. Monitors are unidirectional and won't propagate the errors, but only report it. I guess you could write your own mutual supervision that way, but it would be a bit weird, still.

regularfry 15y ago
My problem with having a tree is that it implies that at every level - including at the root - there's a single point of failure. I get the error kernel concept, I just don't necessarily trust the hardware.
- mononcqc 15y ago
  What you need to do then is look into a few different options. The first one is distributed Applications. This is part of the standard OTP stuff. What this does is declare a node as a master, and then failover nodes for your OTP application. When your master goes down, the other nodes take over it. Whenever the master is back, the failover nodes drop their applications and restarts it on the master. This loses all of the state if it hasn't been kept in a safe place. Another one is to look at gen_leader. gen_leader is not part of OTP, but has been written by Ulf Wiger and is available on github or through agner. A quick description of it would simply be to say that it's a boosted gen_server that will find a master of itself somewhere and automatically replicate its data everywhere else. If the master dies, a leader election occurs and a new one takes over. A third option is to have a masterless concept. This is what happens in Riak. Data has to be shared in clever ways and code organized to support such a design. Nobody's a master, everyone's a follower. You add redundancy here and there, push important state to a safe spot, etc. In the standard library, modules such as 'global' handles this kind of thing. Supervisors are not exactly made to cross node boundaries. They protect your local processes. If you need something more complex to keep your state alive, this usually happens at a higher level for OTP.
  - regularfry 15y ago
    Yeah, it sounds like either distributed applications (more likely 'cos it's in OTP) or gen_leader (maybe interesting technically) would be good to look at. That being said, riak_core has been on my List Of Things To Have A Really Good Look At for a while now...
lenary 15y ago
doesn't heart kinda solve the cyclic "who restarts who thing". I think heart (if enabled) sits there until the erlang process exits, restarts the erlang vm, which in turn also brings up an instance of heart. yeah, complicated and not perfect, but it works.
- mononcqc 15y ago
  Heart does solve it in a way, yes. Any Master/Slave pattern where a masterless slave becomes a master also solves it. The complex cases I had in mind happen when you dynamically add more workers and try to set up crazier links (A is the master of B, B is the master of C, C is the master of A), although there are few practical uses of that. You'd have to try to shoot yourself in the foot I guess. I see it as a bit hard to provide a generic OTP pattern to do that kind of thing past the standard takeover/failover setup (and heart for VMs)

regularfry 15y ago

My problem with having a tree is that it implies that at every level - including at the root - there's a single point of failure. I get the error kernel concept, I just don't necessarily trust the hardware.

mononcqc 15y ago
What you need to do then is look into a few different options. The first one is distributed Applications. This is part of the standard OTP stuff. What this does is declare a node as a master, and then failover nodes for your OTP application. When your master goes down, the other nodes take over it. Whenever the master is back, the failover nodes drop their applications and restarts it on the master. This loses all of the state if it hasn't been kept in a safe place. Another one is to look at gen_leader. gen_leader is not part of OTP, but has been written by Ulf Wiger and is available on github or through agner. A quick description of it would simply be to say that it's a boosted gen_server that will find a master of itself somewhere and automatically replicate its data everywhere else. If the master dies, a leader election occurs and a new one takes over. A third option is to have a masterless concept. This is what happens in Riak. Data has to be shared in clever ways and code organized to support such a design. Nobody's a master, everyone's a follower. You add redundancy here and there, push important state to a safe spot, etc. In the standard library, modules such as 'global' handles this kind of thing. Supervisors are not exactly made to cross node boundaries. They protect your local processes. If you need something more complex to keep your state alive, this usually happens at a higher level for OTP.
- regularfry 15y ago
  Yeah, it sounds like either distributed applications (more likely 'cos it's in OTP) or gen_leader (maybe interesting technically) would be good to look at. That being said, riak_core has been on my List Of Things To Have A Really Good Look At for a while now...

lenary 15y ago

doesn't heart kinda solve the cyclic "who restarts who thing". I think heart (if enabled) sits there until the erlang process exits, restarts the erlang vm, which in turn also brings up an instance of heart. yeah, complicated and not perfect, but it works.

mononcqc 15y ago
Heart does solve it in a way, yes. Any Master/Slave pattern where a masterless slave becomes a master also solves it. The complex cases I had in mind happen when you dynamically add more workers and try to set up crazier links (A is the master of B, B is the master of C, C is the master of A), although there are few practical uses of that. You'd have to try to shoot yourself in the foot I guess. I see it as a bit hard to provide a generic OTP pattern to do that kind of thing past the standard takeover/failover setup (and heart for VMs)