3 ms·

A condition that "should not happen" might still be a problem specific to a particular request. If you "just crash" it turns this request from one that only tri

A condition that "should not happen" might still be a problem specific to a particular request. If you "just crash" it turns this request from one that only triggers a http 500 response to one that crashes the process. This increases the risk of Query of Death scenarios where the frontend that needs to serve this particular request starts retrying it with different backends and triggers restarts faster than the processes come back up. So being too eager to "just crash" may turn a scenario where you fail to serve 1% of requests into a scenario where you serve none because all your processes keep restarting.

zwnow 1y ago
This is funny given Elixir/Erlangs whole idea is "let it crash". In Go I just have a Recovery Middleware for any type of problem. Don't know how other langs do it tho
- knome 1y ago
  erlang doesn't crash the program, it crashes the thread. erlang has a layered management system built in as part of OTP (open telecom platform, erlang was built for running highly concurrent telephony hardware). when a thread crashes, it dies and signals its parent. the parent then decides what to do. usually, that's just restarting the worker. maybe if ten workers have crashed in a minute, the manager itself will die and restart. issues bubble up, and managers restart subsystems automatically. for some things, like parsing user data, you might never cause the manager to die, and just always restart the worker. the article, if you should choose to read it, is explaining that people have the misconception you appear to be having due to the 'let it fail' catchphrase. it goes into detail about this system, when failing is appropriate, and when trying to work around errors is appropriate. as erlang uses greenthreads, restarting a thread for a user API is effectively instant and free.
  - zwnow 1y ago
    It's not a misconception given that Elixir Forum and its Discords members will say that to you. Also I never assumed the whole program crashed so why would you explain this to me? Why would one Blog guy know it better than a lot of other Elixir devs?
    - borromakot 1y ago
      Blog guy here: I do, in fact, know it better than a lot of other Elixir devs.
    - sarchertech 1y ago
      It’s well known among elixir devs that for reasons unkown, Elixir Forum is populated predominantly by people who don’t know what they’re talking about.
- davidclark 1y ago
  I don’t know Go, but that sounds like someone has simply written part of Erlang in Go.
cyberax 1y ago
"Let it crash" in Erlang/Elixir means that the process that serves the request is allowed to crash. It then will be restarted by the supervisor. Supervisors themselves form a tree, so for a crash to take down the whole app, it needs to propagate all the way to the top. Another explanation for people familiar with exceptions in other languages: "Don't try to catch the exception inside a request handler".
josevalim 1y ago
Processes can be marked as temporary, which means they are not restarted, and that’s what is used when managing http connections, as you can’t really restart a request on the server without the client. So the scenario above wouldn’t happen. You still want those processes to crash though, as it allows it to automatically clean up any concurrent work. For example, if during a request you start three processes to do concurrent work, like fetching APIs, then the request process crashes, the concurrent processes are automatically cleaned up.
rtpg 1y ago
My impression is that in Erlang land each process handler is really cheap so you can just keep on showing up with process handlers and not reach exhaustion like you do with other systems (at least in pre-async worlds...)
davidclark 1y ago
You should try to do some load testing of a real Erlang system and compare how it handles this scenario against other languages/frameworks. What you are describing is one of the exact things the Erlang system is strong against due to the scheduler.
sarchertech 1y ago
> If you "just crash" it turns this request from one that only triggers a http 500 response to one that crashes the process. In phoenix each request has its own process and crashing that process will result in a 500 being sent to the client.

zwnow 1y ago

This is funny given Elixir/Erlangs whole idea is "let it crash". In Go I just have a Recovery Middleware for any type of problem. Don't know how other langs do it tho

knome 1y ago
erlang doesn't crash the program, it crashes the thread. erlang has a layered management system built in as part of OTP (open telecom platform, erlang was built for running highly concurrent telephony hardware). when a thread crashes, it dies and signals its parent. the parent then decides what to do. usually, that's just restarting the worker. maybe if ten workers have crashed in a minute, the manager itself will die and restart. issues bubble up, and managers restart subsystems automatically. for some things, like parsing user data, you might never cause the manager to die, and just always restart the worker. the article, if you should choose to read it, is explaining that people have the misconception you appear to be having due to the 'let it fail' catchphrase. it goes into detail about this system, when failing is appropriate, and when trying to work around errors is appropriate. as erlang uses greenthreads, restarting a thread for a user API is effectively instant and free.
- zwnow 1y ago
  It's not a misconception given that Elixir Forum and its Discords members will say that to you. Also I never assumed the whole program crashed so why would you explain this to me? Why would one Blog guy know it better than a lot of other Elixir devs?
  - borromakot 1y ago
    Blog guy here: I do, in fact, know it better than a lot of other Elixir devs.
  - sarchertech 1y ago
    It’s well known among elixir devs that for reasons unkown, Elixir Forum is populated predominantly by people who don’t know what they’re talking about.
davidclark 1y ago
I don’t know Go, but that sounds like someone has simply written part of Erlang in Go.

cyberax 1y ago

"Let it crash" in Erlang/Elixir means that the process that serves the request is allowed to crash. It then will be restarted by the supervisor. Supervisors themselves form a tree, so for a crash to take down the whole app, it needs to propagate all the way to the top. Another explanation for people familiar with exceptions in other languages: "Don't try to catch the exception inside a request handler".

josevalim 1y ago

Processes can be marked as temporary, which means they are not restarted, and that’s what is used when managing http connections, as you can’t really restart a request on the server without the client. So the scenario above wouldn’t happen. You still want those processes to crash though, as it allows it to automatically clean up any concurrent work. For example, if during a request you start three processes to do concurrent work, like fetching APIs, then the request process crashes, the concurrent processes are automatically cleaned up.

rtpg 1y ago

My impression is that in Erlang land each process handler is really cheap so you can just keep on showing up with process handlers and not reach exhaustion like you do with other systems (at least in pre-async worlds...)

davidclark 1y ago

You should try to do some load testing of a real Erlang system and compare how it handles this scenario against other languages/frameworks. What you are describing is one of the exact things the Erlang system is strong against due to the scheduler.

sarchertech 1y ago

> If you "just crash" it turns this request from one that only triggers a http 500 response to one that crashes the process. In phoenix each request has its own process and crashing that process will result in a 500 being sent to the client.