3 ms·

I built a distributed system once on two principles: 1. let it crash, 2. make everything deterministic. obviously, this resulted in crashes being invisible and

• jerf 3y ago
  The way it builds on immutability means it naturally can lean in that direction, but the way it tends to be used for networks a lot undoes that because network communication by its nature is not deterministic in the sense you mean. In my case, IIRC, it was something to the effect of, a lot of our old clients out in the field connected with a field that crashed the handler. Enough of these were connecting that the supervisor was flipping out and restarting things (even working things) because it thought there was a systemic issue. (I mean, it was right, though I had it configured to be a bit to aggressive in its response.) The fact that I could let things crash didn't rescue the fact my system, even if I fixed that config, would strain its resources constantly doing TLS negotiations and then immediately crashing out what were supposed to be long term connections. Obviously, the problem was in the test suite, we were able to roll back, address the problem, ultimately this was a blip not a catastrophe. I just cite it as a case where "let it crash" didn't save me, because the thing that was crashing was just too broken. You still have to write good code in Erlang. It may survive not having "great" code but it's not a license to not care.