3 ms·

I don't think a central exception handler means ONLY handle exception in one very specific point. It means that if you can handle an error handle it, otherwise

I don't think a central exception handler means ONLY handle exception in one very specific point. It means that if you can handle an error handle it, otherwise let the central exception handler handle it. I really like Elixir and the Erlang VM but supervisor trees are not a perfect solution either. For all the talk about how in Erlang you just let things fail and don't have to do defensive coding, any non-trivial application requires a lot of very thorough design on how to handle error scenarios and keep your application consistent. If a process fails too many times too often (say your web service it's trying to ping is inaccessible) it can take out your whole supervisor. If that supervisor is holding an ETS table that holds important coordination data (but not data that requires persistence) that ETS table (and all it's data) is now completely gone, and unless that case has been handled you have a lot of processes running but not doing what they should be doing, and your application is inconsistent until you completely shut it down and restart it. And if you don't plan for a lot of these scenarios then you will be caught completely surprised when it happens in production.

regulation_d 9y ago
If you need an ETS table to have greater resiliency move it up higher in the supervision tree. Nothing prevents you from starting it at the application level. Also you have a lot of control of your supervision tree. You can specify the quantity and frequency of failures that would cause a supervisor to fail. For some use cases, 3 failures in 5 seconds might be reason enough to restart that supervisor. For other use cases, a much higher failure rate might be justified.
- KallDrexx 9y ago
  Right, so you have to make a lot of decisions on what things can fail, how often they should be allowed to fail, what side effects can be tolerated with a major failure, what happens if a bad process fails when it comes back up (thus risking causing the supervisor itself to cascade restart itself), etc... These decisions are not trivial and are exacerbated because of the Actor style of flow due to the independent pieces. So from my experience you have non-BEAM languages where a single error can take down your whole application if you don't plan your failure scenarios properly, where as in BEAM languages if you don't plan your failure scenarios properly you are more likely to end up with an inconsistent system. The former isn't that bad in a lot of circumstances as you should have an external application monitoring and restarting a critical server that's no longer running (even for BEAM systems) but the latter can be very bad because it can be a silent issue. I have systems working at my job written in .net that have been running for over 6 months without a restart, even when Azure had failures and it could not talk to its command and control server for an hour and kept running perfectly fine with using a well thought out architecture and a well placed central error handler for generic errors without things cascading throughout the whole system. So resiliency has less to do with language or supervisor trees and more to do with making every step has failure scenarios thought about and planned for.

regulation_d 9y ago

If you need an ETS table to have greater resiliency move it up higher in the supervision tree. Nothing prevents you from starting it at the application level. Also you have a lot of control of your supervision tree. You can specify the quantity and frequency of failures that would cause a supervisor to fail. For some use cases, 3 failures in 5 seconds might be reason enough to restart that supervisor. For other use cases, a much higher failure rate might be justified.

KallDrexx 9y ago
Right, so you have to make a lot of decisions on what things can fail, how often they should be allowed to fail, what side effects can be tolerated with a major failure, what happens if a bad process fails when it comes back up (thus risking causing the supervisor itself to cascade restart itself), etc... These decisions are not trivial and are exacerbated because of the Actor style of flow due to the independent pieces. So from my experience you have non-BEAM languages where a single error can take down your whole application if you don't plan your failure scenarios properly, where as in BEAM languages if you don't plan your failure scenarios properly you are more likely to end up with an inconsistent system. The former isn't that bad in a lot of circumstances as you should have an external application monitoring and restarting a critical server that's no longer running (even for BEAM systems) but the latter can be very bad because it can be a silent issue. I have systems working at my job written in .net that have been running for over 6 months without a restart, even when Azure had failures and it could not talk to its command and control server for an hour and kept running perfectly fine with using a well thought out architecture and a well placed central error handler for generic errors without things cascading throughout the whole system. So resiliency has less to do with language or supervisor trees and more to do with making every step has failure scenarios thought about and planned for.