4 ms·
I'm a full time elixir dev (and fanboy) but don't agree with your second point. The only thing "easy" about erlang supervisors is having short-lived DB network
by latch 8y ago
I'm a full time elixir dev (and fanboy) but don't agree with your second point.
The only thing "easy" about erlang supervisors is having short-lived DB network hiccup cascade up your supervisor tree and shutting down your app. Clearly supervisors were designed for a set of problems which don't align with what I imagine most Elixir devs are likely to run into.
A lot of people end up building either a special supervisor with infinite retry + backoff, or bake the logic directly into their process, specifically to avoid triggering the built-in supervisor.
If you want a system that runs for years, you're almost certainly going to need an external language-agnostic supervisor (god, upstart, supervisord, docker, ...) At that point, the built-in Supervisor advantage is...overstated.
What isn't overstated is the advantage isolated processes have when it come to managing complexity. Fundamental shift in how technical debt can be made more or less a non-factor.
- dnautics 8y ago> is having short-lived DB network hiccup cascade up your supervisor tree and shutting down your app Can you link me to a description of this? I am currently working on software that expects DB tx on the order of 10/second (so I have literally never had this problem), but am transitioning to a project where I expect to have 10k-100k tx/second. What causes the hiccup, and why is it not taken care of with sensible defaults in the typical libraries (Ecto EG). I'd love to have an ieda of what could be coming down the pike for me (to be defensive) an external language-agnostic supervisor (god, upstart, supervisord, docker, ...) So some of my coworkers are old C++ programmers that were FAANG hotshots and imagine it possible to do everything in C++. They don't write unit tests, have just learned go "because it's better" and don't know kubernetes. There is literally no "high uptime/reliability" story in-house ATM.
- latch 8y agoMore generally: A process (P) fails. Its supervisor (S1) restarts it. If (P) fails 3 times in 5 seconds (configurable,but no backoffs) then (S1) will fail. S1's parent supervisor (S2) will now restart S1, which will restart (P) which might still fail. If P fails quickly enough, you'll cascade your "restart max of X times in Y seconds" all the way up to the application, the ultimate supervisor, which itself will shut down after 3 failures. For DBs specifically, DBConnection relies on trapping exit and reconnecting with a backoff (it does not rely on supervisors), but how YOUR code deals with (or, doesn't deal with) a failure can result in a cascade: defmodule MyProcess do use GenServer ... def something(data) do GenServer.cast(__MODULE__, {:something, data}) end def handle_cast({:something, data}, state) do ... Postgrex.query!("lower case sql because we aren't monsters", []) ... end end If your DB goes down, this process will crash when something/1 is called. If something/1 is called at a rate greater than the supervisors are configured to accept, it'll take down the app.
- macintux 8y agoOne useful piece of advice I recall, perhaps from Mahesh Paolini-Subramanya, was to write defensive code for predictable errors, despite the happy path coding that Erlang allows. So losing connectivity to a database should not result in a process failure, and thus is something that a supervisor should never have to deal with. You’re right that there is no silver bullet internal to the language. Even the BEAM itself can fail.
- yellowapple 8y agoBetter yet: add another layer of supervision. Erlang processes are cheap. Might as well take advantage of that.
- dnautics 8y agoThanks! That's really helpful!
- walterstucco 8y agoJust don't let it crash at a rate greater than the supervisors are configured to accept. It's easy enough If you let it crash when it is expected to crash (not handling the exception) it's your fault Supervisors are for unexpected crashes, they are not a way to propagate exception handling to the parent process
- mietek 8y agoNote HN has hellbanned your account, and your comments are showing up as dead.
- walterstucco 8y agoThanks I know, I've been having this problem for over a year know, they refuse to unban me because I've said a few times something bad about US and refused to take it back Last "offending" post is from nov 2017, on June they were still "angry" at me, eventually I gave up, I don't believe there is free speech here, more than in any other social network I'm keeping it anyway, I got tons of posts saved on this account Anyway, I'm using this opportunity to post their answer From: Scott Bell <scott@ycombinator.com> Wed, Jun 13, 2018, 8:38 PM Posts that are dead can be vouched for by the community to make them visible and replyable. Perhaps you can let them know?
- whizzkid 8y agoI am not an expert on the language but wrote couple of Elixir/Erlang applications. I would love to learn more about the problem you are describing. Using Elixir umbrella, isn't each child application on their own and you don't end up with a monolithic structure? Edit: I think I understand what you mean now. Would you be reading this answer if you have time? https://stackoverflow.com/a/48823641/1238090 https://stackoverflow.com/a/48823641/1238090