5 ms·

To me, it's unclear what the best solution is here. Other languages solve this differently with tradeoffs, e.g. Java defaults to threads silently dying when an

by cle 3y ago

To me, it's unclear what the best solution is here. Other languages solve this differently with tradeoffs, e.g. Java defaults to threads silently dying when an exception isn't caught. Your program will continue to run, but it's probably in some undefined state at that point. There are mechanisms for propagating exceptions elsewhere, but they have to be explicitly set up (like in Go). You can set a default uncaught exception handler, but that's effectively a global variable with all the subsequent "fun", and the uncaught exception handler has to know how to clean up and restore state if the exception was thrown from anywhere, which seems generally difficult to do correctly.

brentjanderson 3y ago
Erlang/Elixir have a great story here: “let it crash”. Because each slice of activity in an application is wrapped in its own process (think single threaded loop but you can run a million at a time, almost free to create and destroy), if it crashes it only takes down that web request/process. Recovery mechanisms are built in to get back to a know good state.
- revskill 3y ago
  Pm2 for nodejs is the same.
- cle 3y ago
  I haven't used Erlang extensively, what happens if you crash in the middle of e.g. holding a lock, or during a coordinated dance with other processes? My concern isn't really "does the program keep running?", it's "does the program keep running correctly?".
  - iudqnolq 3y ago
    The key to Erlang error handling is that crashes should bubble up to a high level which then restarts everything below it in a known good state. If you're in a coordinated dance with another process you link to that process. If a process you're linked to crashes then you crash too. There's no way to block yourself in Erlang such that you can't be told to crash. After you crash your supervisor might restart you, if that's what you configured. Or you might give up on your specific task.
  - brentjanderson 3y ago
    That sort of problem is beyond the scope of the runtime in any case, isn't it? In either of the examples you offered (holding a lock, coordinating with other processes), there must be timeouts enforced by the lock or the other processes so that, if something goes wrong, the system isn't waiting for a crashed process to continue the work. Erlang/Elixir do make this pretty easy to manage, including the scenario where the process does recover by reverting back to a known good state. It won't do it for you automatically, but it exposes enough surface area to make problems like that solvable without reaching for a lot of extra tools - it's built into the runtime.
    - cle 3y ago
      > That sort of problem is beyond the scope of the runtime in any case, isn't it? Yes, which is why Go's outright crashing also makes sense to me...both Go and Erlang's behavior seem conceptually the same, with some architectural tradeoffs. It's not really that different for a process to die and restart. If some shared resource reaches an undefined state, then you have to kill everything and reset your state anyway. I suppose Go's behavior lends itself better to "microservices", whereas Erlang's behavior is better suited for "monolith" processes that do a lot of different things. IMO either of these are better than Java's default behavior of silently swallowing the exception and allowing the thread to quietly die.
dboreham 3y ago
> when an exception isn't caught Not catching all exceptions is a glaring P0 bug.
- funcDropShadow 3y ago
  You should almost never catch all exceptions, i.e. Throwable on the JVM. That is one of the few things that Scala really got right. The `catch NonFatal(e) =>` idiom is doing that nicely. It will catch all throwables with a selected set of special cases, e.g. OutOfMemoryException and all the other VirtulMachineErrors. Catching those in a framework lead to extending the time until a crash follows on a serious issue. Crashing early is often beneficial in such a situation. Together with a process watchdog, like systemd, kubernetes, dockerd, whatever crashing early increases the uptime.
paulddraper 3y ago
Node.js changed behaviors over time.

by cle 3y ago

brentjanderson 3y ago

Erlang/Elixir have a great story here: “let it crash”. Because each slice of activity in an application is wrapped in its own process (think single threaded loop but you can run a million at a time, almost free to create and destroy), if it crashes it only takes down that web request/process. Recovery mechanisms are built in to get back to a know good state.

revskill 3y ago
Pm2 for nodejs is the same.
cle 3y ago
I haven't used Erlang extensively, what happens if you crash in the middle of e.g. holding a lock, or during a coordinated dance with other processes? My concern isn't really "does the program keep running?", it's "does the program keep running correctly?".
- iudqnolq 3y ago
  The key to Erlang error handling is that crashes should bubble up to a high level which then restarts everything below it in a known good state. If you're in a coordinated dance with another process you link to that process. If a process you're linked to crashes then you crash too. There's no way to block yourself in Erlang such that you can't be told to crash. After you crash your supervisor might restart you, if that's what you configured. Or you might give up on your specific task.
- brentjanderson 3y ago
  That sort of problem is beyond the scope of the runtime in any case, isn't it? In either of the examples you offered (holding a lock, coordinating with other processes), there must be timeouts enforced by the lock or the other processes so that, if something goes wrong, the system isn't waiting for a crashed process to continue the work. Erlang/Elixir do make this pretty easy to manage, including the scenario where the process does recover by reverting back to a known good state. It won't do it for you automatically, but it exposes enough surface area to make problems like that solvable without reaching for a lot of extra tools - it's built into the runtime.
  - cle 3y ago
    > That sort of problem is beyond the scope of the runtime in any case, isn't it? Yes, which is why Go's outright crashing also makes sense to me...both Go and Erlang's behavior seem conceptually the same, with some architectural tradeoffs. It's not really that different for a process to die and restart. If some shared resource reaches an undefined state, then you have to kill everything and reset your state anyway. I suppose Go's behavior lends itself better to "microservices", whereas Erlang's behavior is better suited for "monolith" processes that do a lot of different things. IMO either of these are better than Java's default behavior of silently swallowing the exception and allowing the thread to quietly die.

dboreham 3y ago

> when an exception isn't caught Not catching all exceptions is a glaring P0 bug.

funcDropShadow 3y ago
You should almost never catch all exceptions, i.e. Throwable on the JVM. That is one of the few things that Scala really got right. The `catch NonFatal(e) =>` idiom is doing that nicely. It will catch all throwables with a selected set of special cases, e.g. OutOfMemoryException and all the other VirtulMachineErrors. Catching those in a framework lead to extending the time until a crash follows on a serious issue. Crashing early is often beneficial in such a situation. Together with a process watchdog, like systemd, kubernetes, dockerd, whatever crashing early increases the uptime.

paulddraper 3y ago

Node.js changed behaviors over time.