5 ms·

The argument here is that the system can get itself into a state that is irreparable (and perhaps very difficult to track / debug). By crashing and having it s

by sb8244 5y ago

The argument here is that the system can get itself into a state that is irreparable (and perhaps very difficult to track / debug). By crashing and having it start into a good state, the system will at least be operational again.

ch4s3 5y ago
As long as you don’t cause a crash loop you’re golden in my experience.
- zelphirkalt 5y ago
  A crash loop would indicate something to be fundamentally broken in ones code. That's OK too, because then we know, we have to fix it.
munchler 5y ago
With all due respect, that does nothing to prevent bugs. In fact, it almost seems to encourage them: "Your process crashed? Don't worry about finding and fixing the bug. We'll just restart it again."
- onlyrealcuzzo 5y ago
  Depends how the scale you're working on, I suppose. If you running a thousands of programs on hundreds of thousands of machines 24-7, you're bound to run into weird edge cases at the system level. Instead of worrying about and optimizing all of these edge cases, SOMETIMES it is better to just have a system that is tolerant of these edge cases by design.
  - munchler 5y ago
    Perhaps, but that's an entirely different claim from the one made in the article (that Erlang can prevent such bugs).
    - macintux 5y ago
      The claim was it prevents Heisenbugs.
      - scns 5y ago
        It does not prevent them. The reasoning is, that those bugs are so obscure, only occuring in vanishingly rare constellations, that finding and preventing them would be prohibitively hard/expensive.
        polack 5y ago
        Say you have an application that takes user input, validates it and then try to add it to the current state. If there is a bug in the validation so that 0 is allowed as input, but later when you try to add it to the current state you divide by 0. In Java you would explicitly handle this case with a catch and take action on a "division by zero exception". In Erlang you would just let the process crash (and log the reason so bug can be fixed) and restart the state to what it was before the erroneous input, no matter what bug/case the wrong input triggered. By having this generic handling you will make a really resilient system since you don't have to handle every possible bug on a case-by-case basis. What would be the advantage of handling this case in the Java defensive programming approach? Maybe someone will catch the error and return/introduce null to the state? Or BigDecimal.ZERO? Then you might end up in an unexpected state for all subsequent requests.
        scns 5y ago
        Well, division by 0 could be handled with pattern matching without crashing. Excuse my rusty pseudoerlang. (edit) formatting handle_zero(data, 0) -> {:error, "division by zero"}, handle_zero(data, _) -> {:ok, transform(data)}. handle_input(data) -> handle_zero(data, data.divider).
- outime 5y ago
  This doesn’t prevent bugs (if we’re talking about logical ones), it prevents extended outages because of them. In other words, if the application crashes you can log such fact and fix it later while your end users won’t notice a problem on your side in the right circumstances (unless it’s a crash loop which can’t be temporarily solved by restarting).
  - dnautics 5y ago
    Or you can log it and never fix it. I noticed a flurry of crashes that came about on restarting the vm. Almost certainly transient startup race conditions. Causes effectively zero service downtime and no degradation of QOS for end user (other nodes in the cluster could service requests). But having a disorderly startup is less of a pain in the butt to program and may even result in a quicker time to start. So who cares. Never fixed it.
  - corinroyal 5y ago
    The point is the app doesn't crash. One process (object) crashes and immediately restarts. The entire rest of the app keeps chugging away. Compare to a language with one or a few main threads, where one bug can crash the whole system. It's a brillant scheme designed for systems required run for a decade or more with only minutes of downtime (telephone switches).
- jlouis 5y ago
  Detecting a bug, crashing, and then restarting, can be much easier to write than code which is bug-free. In fact, some times, you need so much code that it is likely you are introducing more bugs by trying to handle an edge-case. A key point of Erlang systems, however, is that they are really good at reporting the state of the system when it crashes (due to functional programming, you have the state from before the crash happened, and what event lead to the crash). The restart is a stop-gap measure that gives you service for the system as a whole. You can then look at the logged bug report and fix the problem. But you are in control of how quickly you want to fix the problem. There is a cost to fixing a bug as well.
- keithnz 5y ago
  It's more the subtle difference between robustness and correctness.
- jimbokun 5y ago
  Yes, and isn't that amazing! Bugs in production will happen in every language, and crashing one of many running processes and restarting it to get it back into a known good state sure beats having to scramble to find a fix, code it, build and deploy a new version of the code.
- scns 5y ago
  It enables systems with an uptime of 99.nine9s% that means less than 4 seconds <edit>downtime per</edit> year.
- e_proxus 5y ago
  It’s all about resilience, not about preventing bugs completely. The beauty of it all is that you can skip on a lot of error handling code whose only purpose is to keep the system running. You only need to handle business logic errors. Let’s take a web server as an example. The way the “let it crash” philosophy is done in Erlang is that you crash individual processes first (e.g. the current HTTP request from one browser). If that keeps happening there’s a counter that crashes the subsystem (e.g. the web file listing component). If that too keeps crashing the whole web frontend might crash, but the node might still be up (e.g. serving FTP or whatnot). And lastly the node will shut down completely if the web frontend keeps crashing. This way, the rest of the system keeps performing and serving requests even if some parts are not working intermittently. On top of this you of course have (built-in) logging of all these errors so you can investigate them.

by sb8244 5y ago

ch4s3 5y ago

As long as you don’t cause a crash loop you’re golden in my experience.

zelphirkalt 5y ago
A crash loop would indicate something to be fundamentally broken in ones code. That's OK too, because then we know, we have to fix it.

munchler 5y ago

With all due respect, that does nothing to prevent bugs. In fact, it almost seems to encourage them: "Your process crashed? Don't worry about finding and fixing the bug. We'll just restart it again."

onlyrealcuzzo 5y ago
Depends how the scale you're working on, I suppose. If you running a thousands of programs on hundreds of thousands of machines 24-7, you're bound to run into weird edge cases at the system level. Instead of worrying about and optimizing all of these edge cases, SOMETIMES it is better to just have a system that is tolerant of these edge cases by design.
- munchler 5y ago
  Perhaps, but that's an entirely different claim from the one made in the article (that Erlang can prevent such bugs).
  - macintux 5y ago
    The claim was it prevents Heisenbugs.
    - scns 5y ago
      It does not prevent them. The reasoning is, that those bugs are so obscure, only occuring in vanishingly rare constellations, that finding and preventing them would be prohibitively hard/expensive.
      - polack 5y ago
        Say you have an application that takes user input, validates it and then try to add it to the current state. If there is a bug in the validation so that 0 is allowed as input, but later when you try to add it to the current state you divide by 0. In Java you would explicitly handle this case with a catch and take action on a "division by zero exception". In Erlang you would just let the process crash (and log the reason so bug can be fixed) and restart the state to what it was before the erroneous input, no matter what bug/case the wrong input triggered. By having this generic handling you will make a really resilient system since you don't have to handle every possible bug on a case-by-case basis. What would be the advantage of handling this case in the Java defensive programming approach? Maybe someone will catch the error and return/introduce null to the state? Or BigDecimal.ZERO? Then you might end up in an unexpected state for all subsequent requests.
        scns 5y ago
        Well, division by 0 could be handled with pattern matching without crashing. Excuse my rusty pseudoerlang. (edit) formatting handle_zero(data, 0) -> {:error, "division by zero"}, handle_zero(data, _) -> {:ok, transform(data)}. handle_input(data) -> handle_zero(data, data.divider).
outime 5y ago
This doesn’t prevent bugs (if we’re talking about logical ones), it prevents extended outages because of them. In other words, if the application crashes you can log such fact and fix it later while your end users won’t notice a problem on your side in the right circumstances (unless it’s a crash loop which can’t be temporarily solved by restarting).
- dnautics 5y ago
  Or you can log it and never fix it. I noticed a flurry of crashes that came about on restarting the vm. Almost certainly transient startup race conditions. Causes effectively zero service downtime and no degradation of QOS for end user (other nodes in the cluster could service requests). But having a disorderly startup is less of a pain in the butt to program and may even result in a quicker time to start. So who cares. Never fixed it.
- corinroyal 5y ago
  The point is the app doesn't crash. One process (object) crashes and immediately restarts. The entire rest of the app keeps chugging away. Compare to a language with one or a few main threads, where one bug can crash the whole system. It's a brillant scheme designed for systems required run for a decade or more with only minutes of downtime (telephone switches).
jlouis 5y ago
Detecting a bug, crashing, and then restarting, can be much easier to write than code which is bug-free. In fact, some times, you need so much code that it is likely you are introducing more bugs by trying to handle an edge-case. A key point of Erlang systems, however, is that they are really good at reporting the state of the system when it crashes (due to functional programming, you have the state from before the crash happened, and what event lead to the crash). The restart is a stop-gap measure that gives you service for the system as a whole. You can then look at the logged bug report and fix the problem. But you are in control of how quickly you want to fix the problem. There is a cost to fixing a bug as well.
keithnz 5y ago
It's more the subtle difference between robustness and correctness.
jimbokun 5y ago
Yes, and isn't that amazing! Bugs in production will happen in every language, and crashing one of many running processes and restarting it to get it back into a known good state sure beats having to scramble to find a fix, code it, build and deploy a new version of the code.
scns 5y ago
It enables systems with an uptime of 99.nine9s% that means less than 4 seconds <edit>downtime per</edit> year.
e_proxus 5y ago
It’s all about resilience, not about preventing bugs completely. The beauty of it all is that you can skip on a lot of error handling code whose only purpose is to keep the system running. You only need to handle business logic errors. Let’s take a web server as an example. The way the “let it crash” philosophy is done in Erlang is that you crash individual processes first (e.g. the current HTTP request from one browser). If that keeps happening there’s a counter that crashes the subsystem (e.g. the web file listing component). If that too keeps crashing the whole web frontend might crash, but the node might still be up (e.g. serving FTP or whatnot). And lastly the node will shut down completely if the web frontend keeps crashing. This way, the rest of the system keeps performing and serving requests even if some parts are not working intermittently. On top of this you of course have (built-in) logging of all these errors so you can investigate them.