8 ms·

On Rigorous Error Handling

67 points by rumcajz 8y ago

sfilargi 8y ago
And because our error handling code rarely, if ever gets executed(I often see trivial mistakes that lead to crashes in error handling case), I think the erlang philosophy of “let it crash” is the best approach IMHO.
- rumcajz 8y ago
  I like to pepper my code with asserts for that reason. But you can also get the case where the process crashes and get restarted in an infinite loop, so it's not a panacea either.
  - macintux 8y ago
    That’s where Erlang’s supervision model comes to the rescue. Restart until it becomes obvious something more serious is wrong and then raise a flag. The nice thing about Erlang is that practically every line of code is an assertion and they’re all live in production. Such a huge advantage over development assertions that get thrown away for prod.
    - MrBuddyCasino 8y ago
      I think Rust got this right, too. No Exceptions, but a Result<R,E> enum return type and the compiler forces you to handle the error case as well by doing exhaustiveness checks.
      - macintux 8y ago
        Doesn't that lead to a lot of error handling code? Another nice feature of the Erlang model is that, often, you can code the happy path and forget the error checking. Makes for much tighter/cleaner/easier-to-read code.
        jjnoakes 8y ago
        In many cases the question mark operator can propagate errors for you. It is both explicit and non-verbose.
  - howard941 8y ago
    I too like to do this yet I feel hobbled by the infinite loop problem on bare iron (watchdog fires off, provided it's not preprocessed out in the release). A sibling brings up the Erlang supervisor and I like failing fast and often when someone's around to see that the log's littered with restarts. Is that the best one can hope for - to leave traces for my successor to pick up the pieces?
- Too 8y ago
  This is how you should use exceptions also. Let it throw. The top level exception handler (aka the supervisor) decides how deep the application is allowed to crash.
makecheck 8y ago
Error-handling code is definitely poorly-tested in my experience across many code bases. Although printf()-style logging has advantages, a huge disadvantage is that it tends to be the reason error-handling code fails: something meant to write a simple log message gets the format/type wrong and an error condition turns into a crash or obscure corruption. In fact, logging code that was once correct can become wrong if the target variable type changes. This is why I love Python format-strings with “{}”, a.k.a. the “just do the right thing here” syntax. Generally the advice of “pick a few failure types and stick to them” is exactly right. You not only encourage error handling to take place but that code is likely to remain correct/complete over time.
- ben-schaaf 8y ago
  An issue that can still happen with python's string formatting is that you can get the number of arguments/`{}` wrong. D (other languages too, probably) has compile-time format string, as well as automatic string conversion: `format!"%s"(2)`. Giving the wrong format string/argument types fails at compile time. Some C/C++ compilers also automatically check this for printf. Though they can still fail if the actual output writing fails, ie. stdout is closed or doesn't exist.
  - teddyh 8y ago
    > An issue that can still happen with python's string formatting is that you can get the number of arguments/`{}` wrong. Easily avoided by using the latest Python feature, added in 3.6: Formatted string literals, a.k.a. f-strings¹. Instead of using "foo {} baz".format(bar) you use f"foo {bar} baz" 1. https://www.python.org/dev/peps/pep-0498 https://www.python.org/dev/peps/pep-0498
- thrower123 8y ago
  This is one of the reasons I love the newer C# string interpolation feature. No more frigging string.Format() strings with positional arguments.
skybrian 8y ago
Provided that you have a way of simulating an error, you can test the corresponding error-handling path. Then these paths will get executed whenever you run your test suite. A coverage tool can be used to find any error handling code that wasn't tested. (But it won't help you find error handling that's missing altogether.)
hnmonkey 8y ago
This might be a shot in the dark but does anyone know how to test error handling in Rails? Say for example you have a method that has a begin block with 10 lines of code in it and a rescue that does some logging or does a puts statement. How do you write a spec that tests the code within the rescue automatically without modifying the 10 lines of code? My google-fu on this one has failed.
- Perseids 8y ago
  I'm probably misunderstanding your situation, but why don't you (from the test suite) mock some function used in the begin block to throw an error and then `expect(the_logging_function).to have_received` the logging output?
  - hnmonkey 8y ago
    That's not actually a bad idea at all. What however if you're just doing simple variables manipulations in those 10 lines and not calling methods (I realize that you can override the math methods as well)? Is there a way to handle that easily or inject an exception into the begin block?
teddyh 8y ago
With exceptions being classes, a library designer can vastly simplify it for the users by creating useful class hierarchies for the exceptions, allowing the user to be as specific or generic as they wish or need.
- rumcajz 8y ago
  That's the theory. In reality each layer just passes exceptions from lower layers on, leaving the end user with a mess that cannot be handled in any sane way.
  - he0001 8y ago
    To me that’s an indication that those lower layers aren’t lower layers.
- hliyan 8y ago
  I've been writing code for 15 years (in Java, C++ and more recently JavaScript and Go). I have finally given up on exceptions and settled on return values (I particularly like Go's system). Exceptions, while theoretically superior, simply tempts even good programmers to just kick errors down the callstack. I prefer guard clauses [1]. [1] http://wiki.c2.com/?GuardClause http://wiki.c2.com/?GuardClause
  - dtech 8y ago
    I like the functional approach with Try, basically the exception becomes part of the return type, and the code chooses to either return an exception or the actual value. fun doSomething(arg: X): Try<Y> Since the return signature needs adjusting, this leads to developers very consciously making the choice to either handle the error in the function, therefore avoiding adjusting the return type, or let the caller deal with it if it isn't logical to handle the error there.
    - wool_gather 8y ago
      Is this the same as `Result<Y>` (where `Result` is a sum type that contains either a value of `Y` or an "Error")? I haven't see it called `Try` before.
nyc111 8y ago
Reading the article made me think that this is how medical doctors work. All diseases are codified and symptoms are errors. Doctors try to match the error to the error code and take action by prescribing codified medications.
- windows_tips 8y ago
  https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system
- anticensor 8y ago
  This is how diagnosers work, which is an important part but not entirety of medical doctors. Research medicine is an extant thing.
fmap 8y ago
> When possible errors are part of the function specification, on the other hand, we are almost OK. This is the single best piece of advice in this article. The second thing you have to document is the postcondition in case of an error - what state is the program left in? With both a normal and an error postcondition you can fully specify your program. Like the author, I'm convinced that most of the pain with error handling stems from programmers ignoring the consequences of an error. That's the reason why approaches that force you to deal with errors explicitly (Maybe a, Result e a, etc.) end up being more robust. Otherwise, a part of the program just ends up missing. However, from a theoretical perspective, exceptions are superior. The reason is that the error postcondition really does represent a non-local exit. Just like an ordinary return statement, it should be implemented as one instead of forcing programmers to walk the stack by hand. The latter is both less efficient and more error prone. Additionally, resource management must be integrated with error handling anyway and exceptions provide a clean opportunity to connect the two. This is one of the things that C++ gets right.
- rumcajz 8y ago
  Is there a way to reconcile the two? If the stack is walked up automatically programmers aren't going to deal with errors explicitly. No?
- panic 8y ago
  When a database transaction fails, it rolls back to the state before the transaction. Exceptions ought to work like this too. Then you wouldn’t have to think about all the places an exception could be thrown. A try-catch block would either completely succeed, following the well-tested success path, or completely fail, leaving the program in its original state.
  - gpderetta 8y ago
    In C++ there is a thing called exception safety and there are three level: * No throw: the function will not fail. Full stop. * Strong exception safety: if the function fails the state of the object(s) is acting on is unchanged. This is similar to transactional atomicity guarantee. * Basic guarantee. If the function fails the state of the objects is unspecified but valid (i.e. no invariant is violated), but data might be lost. From the point of view of the caller of course no throw is the most desirable property, the strong and finally basic. Anything less than that (i.e. corruption, leaks, dangling pointers) is considered unacceptable. Another important insight is realising that exception guarantees have little to do with exceptions and everything to do with postconditions in the return path: for example the same techniques used to guarantee strong safety on the face of exceptions also work to guarantee postconditions on the faceof multiple explicit retun paths.
ken 8y ago
> Programmers want to implement new features. Writing error handling is just an annoyance that slows them down. For me, personally, this is backwards. As a programmer, I want to write error handling (especially for infrastructure), because it means I'll be able to work more quickly later. I won't have to debug through all these abstraction layers. It's the manager who always says "It (the demo = happy path) looks good, so it's time to move on to the next feature".

67 points by rumcajz 8y ago

And because our error handling code rarely, if ever gets executed(I often see trivial mistakes that lead to crashes in error handling case), I think the erlang philosophy of “let it crash” is the best approach IMHO.

rumcajz 8y ago
I like to pepper my code with asserts for that reason. But you can also get the case where the process crashes and get restarted in an infinite loop, so it's not a panacea either.
- macintux 8y ago
  That’s where Erlang’s supervision model comes to the rescue. Restart until it becomes obvious something more serious is wrong and then raise a flag. The nice thing about Erlang is that practically every line of code is an assertion and they’re all live in production. Such a huge advantage over development assertions that get thrown away for prod.
  - MrBuddyCasino 8y ago
    I think Rust got this right, too. No Exceptions, but a Result<R,E> enum return type and the compiler forces you to handle the error case as well by doing exhaustiveness checks.
    - macintux 8y ago
      Doesn't that lead to a lot of error handling code? Another nice feature of the Erlang model is that, often, you can code the happy path and forget the error checking. Makes for much tighter/cleaner/easier-to-read code.
      - jjnoakes 8y ago
        In many cases the question mark operator can propagate errors for you. It is both explicit and non-verbose.
- howard941 8y ago
  I too like to do this yet I feel hobbled by the infinite loop problem on bare iron (watchdog fires off, provided it's not preprocessed out in the release). A sibling brings up the Erlang supervisor and I like failing fast and often when someone's around to see that the log's littered with restarts. Is that the best one can hope for - to leave traces for my successor to pick up the pieces?
Too 8y ago
This is how you should use exceptions also. Let it throw. The top level exception handler (aka the supervisor) decides how deep the application is allowed to crash.

makecheck 8y ago

Error-handling code is definitely poorly-tested in my experience across many code bases. Although printf()-style logging has advantages, a huge disadvantage is that it tends to be the reason error-handling code fails: something meant to write a simple log message gets the format/type wrong and an error condition turns into a crash or obscure corruption. In fact, logging code that was once correct can become wrong if the target variable type changes. This is why I love Python format-strings with “{}”, a.k.a. the “just do the right thing here” syntax. Generally the advice of “pick a few failure types and stick to them” is exactly right. You not only encourage error handling to take place but that code is likely to remain correct/complete over time.

ben-schaaf 8y ago
An issue that can still happen with python's string formatting is that you can get the number of arguments/`{}` wrong. D (other languages too, probably) has compile-time format string, as well as automatic string conversion: `format!"%s"(2)`. Giving the wrong format string/argument types fails at compile time. Some C/C++ compilers also automatically check this for printf. Though they can still fail if the actual output writing fails, ie. stdout is closed or doesn't exist.
- teddyh 8y ago
  > An issue that can still happen with python's string formatting is that you can get the number of arguments/`{}` wrong. Easily avoided by using the latest Python feature, added in 3.6: Formatted string literals, a.k.a. f-strings¹. Instead of using "foo {} baz".format(bar) you use f"foo {bar} baz" 1. https://www.python.org/dev/peps/pep-0498 https://www.python.org/dev/peps/pep-0498
thrower123 8y ago
This is one of the reasons I love the newer C# string interpolation feature. No more frigging string.Format() strings with positional arguments.

skybrian 8y ago

Provided that you have a way of simulating an error, you can test the corresponding error-handling path. Then these paths will get executed whenever you run your test suite. A coverage tool can be used to find any error handling code that wasn't tested. (But it won't help you find error handling that's missing altogether.)

hnmonkey 8y ago

This might be a shot in the dark but does anyone know how to test error handling in Rails? Say for example you have a method that has a begin block with 10 lines of code in it and a rescue that does some logging or does a puts statement. How do you write a spec that tests the code within the rescue automatically without modifying the 10 lines of code? My google-fu on this one has failed.

Perseids 8y ago
I'm probably misunderstanding your situation, but why don't you (from the test suite) mock some function used in the begin block to throw an error and then `expect(the_logging_function).to have_received` the logging output?
- hnmonkey 8y ago
  That's not actually a bad idea at all. What however if you're just doing simple variables manipulations in those 10 lines and not calling methods (I realize that you can override the math methods as well)? Is there a way to handle that easily or inject an exception into the begin block?

teddyh 8y ago

With exceptions being classes, a library designer can vastly simplify it for the users by creating useful class hierarchies for the exceptions, allowing the user to be as specific or generic as they wish or need.

rumcajz 8y ago
That's the theory. In reality each layer just passes exceptions from lower layers on, leaving the end user with a mess that cannot be handled in any sane way.
- he0001 8y ago
  To me that’s an indication that those lower layers aren’t lower layers.
hliyan 8y ago
I've been writing code for 15 years (in Java, C++ and more recently JavaScript and Go). I have finally given up on exceptions and settled on return values (I particularly like Go's system). Exceptions, while theoretically superior, simply tempts even good programmers to just kick errors down the callstack. I prefer guard clauses [1]. [1] http://wiki.c2.com/?GuardClause http://wiki.c2.com/?GuardClause
- dtech 8y ago
  I like the functional approach with Try, basically the exception becomes part of the return type, and the code chooses to either return an exception or the actual value. fun doSomething(arg: X): Try<Y> Since the return signature needs adjusting, this leads to developers very consciously making the choice to either handle the error in the function, therefore avoiding adjusting the return type, or let the caller deal with it if it isn't logical to handle the error there.
  - wool_gather 8y ago
    Is this the same as `Result<Y>` (where `Result` is a sum type that contains either a value of `Y` or an "Error")? I haven't see it called `Try` before.

nyc111 8y ago

Reading the article made me think that this is how medical doctors work. All diseases are codified and symptoms are errors. Doctors try to match the error to the error code and take action by prescribing codified medications.

windows_tips 8y ago
https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system
anticensor 8y ago
This is how diagnosers work, which is an important part but not entirety of medical doctors. Research medicine is an extant thing.

fmap 8y ago

> When possible errors are part of the function specification, on the other hand, we are almost OK. This is the single best piece of advice in this article. The second thing you have to document is the postcondition in case of an error - what state is the program left in? With both a normal and an error postcondition you can fully specify your program. Like the author, I'm convinced that most of the pain with error handling stems from programmers ignoring the consequences of an error. That's the reason why approaches that force you to deal with errors explicitly (Maybe a, Result e a, etc.) end up being more robust. Otherwise, a part of the program just ends up missing. However, from a theoretical perspective, exceptions are superior. The reason is that the error postcondition really does represent a non-local exit. Just like an ordinary return statement, it should be implemented as one instead of forcing programmers to walk the stack by hand. The latter is both less efficient and more error prone. Additionally, resource management must be integrated with error handling anyway and exceptions provide a clean opportunity to connect the two. This is one of the things that C++ gets right.

rumcajz 8y ago
Is there a way to reconcile the two? If the stack is walked up automatically programmers aren't going to deal with errors explicitly. No?
panic 8y ago
When a database transaction fails, it rolls back to the state before the transaction. Exceptions ought to work like this too. Then you wouldn’t have to think about all the places an exception could be thrown. A try-catch block would either completely succeed, following the well-tested success path, or completely fail, leaving the program in its original state.
- gpderetta 8y ago
  In C++ there is a thing called exception safety and there are three level: * No throw: the function will not fail. Full stop. * Strong exception safety: if the function fails the state of the object(s) is acting on is unchanged. This is similar to transactional atomicity guarantee. * Basic guarantee. If the function fails the state of the objects is unspecified but valid (i.e. no invariant is violated), but data might be lost. From the point of view of the caller of course no throw is the most desirable property, the strong and finally basic. Anything less than that (i.e. corruption, leaks, dangling pointers) is considered unacceptable. Another important insight is realising that exception guarantees have little to do with exceptions and everything to do with postconditions in the return path: for example the same techniques used to guarantee strong safety on the face of exceptions also work to guarantee postconditions on the faceof multiple explicit retun paths.

ken 8y ago

> Programmers want to implement new features. Writing error handling is just an annoyance that slows them down. For me, personally, this is backwards. As a programmer, I want to write error handling (especially for infrastructure), because it means I'll be able to work more quickly later. I won't have to debug through all these abstraction layers. It's the manager who always says "It (the demo = happy path) looks good, so it's time to move on to the next feature".