6 ms·

Handling exceptions in Python like a pro

46 points by latrova 5y ago

nighthawk454 5y ago
Some good practices, reminds of Effective Java. Couple nuggets as well: * just `raise` implies re-raising the current exception * `raise MyException() from ex` continues the stack trace * `logger.info(..., exec_info=True)` can add a stack trace to non-error level logs
- stevula 5y ago
  Exception chaining (`raise from`) was new to me but after reading the docs it seems to be the default behavior in except/finally blocks, so the examples from the article don’t actually add anything. It doesn’t hurt to be explicit though. https://docs.python.org/3/tutorial/errors.html#exception-chaining https://docs.python.org/3/tutorial/errors.html#exception-cha...
  - Doxin 5y ago
    Using "raise foo from bar" changes the message between the exceptions from "During handling of the above exception, another exception occurred:" to "The above exception was the direct cause of the following exception:" So yes, these days the whole "raise from" construct is nearly pointless. Used to be that exception chaining wasn't automatic though.
  - aidos 5y ago
    I believe there’s a subtle difference in the meanings. You can raise or raise from. In either case the chaining means you get to see both messages. But one implies the original error is the cause. It’s so subtle that I don’t recall which is which!
Animats 5y ago
try: ... except Exception as e: Don't catch Exception at a low level. Catch OSError, which now includes EnvironmentError and IOError. This should cover all the things that can occur for external reasons. If you get some unexpected error such as a SyntaxError or TypeError, for example, that indicates a bug in the program. At the outermost level, catch Exception, and log a bug with a backtrace if an unhandled exception reaches that level. An exception is not an error code.
bedobi 5y ago
Some copy paste from previous posts Exception based error handling is so bad and unsafe that adopting functional error handling with Either, Try etc as implemented by functional addon libraries for many languages, while not yet common, in time it will become the new default even in OO languages. (just like it's been the default in functional languages for decades) Functional error handling types are much simpler, safer and more powerful. Simpler because they don't rely on dedicated syntax- they're just regular objects no different to any other object. Safer because unlike exceptions, they force callers to handle all potential outcomes, but no more. (no risk of ignoring errors and no risk of catching a higher level of error than desired, ubiquitous bugs in exception based error handling) Powerful because they support map, flatmap, applicative etc, making it easy to eg chain multiple computations together in desired ways, which is unwieldy and bug prone when using exceptions. What is wrong about dedicated syntax It adds complexity to the language! It could be that, when learning Java, Kotlin and any other language, we learn that methods return objects... and that's that. No weird dedicated syntax and magic, special treatment for returning anything other than the happy path, and the HUGE complexity that comes with it, eg the dedicated syntax itself and how it behaves, differences between checked and unchecked exceptions, hierarchies of exceptions etc etc. that makes our lives easier and our work more efficient But that's our point, it doesn't. Exceptions based error handling is unnecessary, hugely complex, doesn't compose at all, obfuscates or straight up hides what can go wrong with any given call, so leads to countless trivially preventable bugs... I could go on. And after decades of use, there's still no consensus about what exceptions should be or how they should be used. Exceptions are a failed experiment and I have no doubt that in ten years, Java, Kotlin and many other languages will acknowledge as much and move away from it the same way Joda Time outcompeted and replaced the horrible Java date and time library.
- nesarkvechnep 5y ago
  I've had great success with `Either` in Typescript. For each layer in the application I created ADTs representing the possible errors and used them in `Either`s. Each layer maps the lower layer errors to it's own.
- nlitened 5y ago
  From what I’ve seen so far, in reality people tend to overuse the `try!` macros, which leads to the same old bubbling exceptions semantic, but more verbose, less performant, and lacking stack traces.
gorpovitch 5y ago
When using a tool like Sentry, you might want to logger.exception(e) instead of a string error message, that way the whole stack trace with helpful debugging information (local variables...) is included in Sentry.
- nicbou 5y ago
  You might also want to use formatted strings too as it allows some logging tools to group the messages together.
- tmarice 5y ago
  logger.exception is basically a shortcut for logger.error(exc_info=True), so there's no need for explicitly assigning e as an error message.
  - aidos 5y ago
    And if you look carefully you’ll find that doing logger.exception(e) will just cast e as a string, which is just the message in the e. So you end up just sending the message twice (along with the stack trace). Instead you can do logger.exception(“something helpful to give a little more context at a glance”)
mtlynch 5y ago
I disagree with most of this advice. I think Google's Python Style Guide provides exception guidance that leads to clearer code.[0] I agree with Animats's comment[1] about not catching the base Exception class, and the Google guidelines explicitly prohibit this, except at the outermost scope. One obvious flaw in catching Exception is if that if the user hits Ctrl+C to kill the application, but the code happens to be executing the ReceiptService at the time, the app will confusingly raise a ReceiptGenerationFailed exception instead of KeyboardInterrupt. (Edit: this isn't true, thanks to child comment for correction) The final code snippet shows four function calls within the same try even though the functions throw a disjoint set of exceptions. It's unclear to a reader at the callsite what exception handlers corrrelate with which functions. except OrderAlreadyInProgress as e: logger.info("Aborting emission request because it is already in progress!") return {"order_id": order_id, "order_status": e.order_status.value} To me, it feels sinful to use an exception to effectively "return" a value that the caller needs. I only use member fields in the exception to describe what went wrong. The original "bad" version of the code returned this information to the caller without raising an exception, and that felt more intuitive to me. else: return {"order_id": order_id, "order_status": order_status.value} I don't understand the purpose of the else at the end. I would just write the return within the try as: return {"order_id": order_id, "order_status": status_service.get_order_status(order_id) } [0] https://google.github.io/styleguide/pyguide.html https://google.github.io/styleguide/pyguide.html [1] https://news.ycombinator.com/item?id=27217301 https://news.ycombinator.com/item?id=27217301
- tmarice 5y ago
  https://docs.python.org/3/library/exceptions.html#KeyboardInterrupt https://docs.python.org/3/library/exceptions.html#KeyboardIn... `KeyboardInterrupt` inherits from `BaseException` instead of `Exception` exactly in order to avoid this situation.
  - mtlynch 5y ago
    Oh, thanks for the correction. I remember running into this myself, but I must have done: except:
- deleted 5y ago
nicbou 5y ago
One more thing: avoid broad catch clauses. Some exceptions ought to break things, like KeyboardInterrupt. By the way, you probably shouldn't wrap code lines on mobile. The code was difficult to read on mobile.

46 points by latrova 5y ago

Some good practices, reminds of Effective Java. Couple nuggets as well: * just `raise` implies re-raising the current exception * `raise MyException() from ex` continues the stack trace * `logger.info(..., exec_info=True)` can add a stack trace to non-error level logs

stevula 5y ago
Exception chaining (`raise from`) was new to me but after reading the docs it seems to be the default behavior in except/finally blocks, so the examples from the article don’t actually add anything. It doesn’t hurt to be explicit though. https://docs.python.org/3/tutorial/errors.html#exception-chaining https://docs.python.org/3/tutorial/errors.html#exception-cha...
- Doxin 5y ago
  Using "raise foo from bar" changes the message between the exceptions from "During handling of the above exception, another exception occurred:" to "The above exception was the direct cause of the following exception:" So yes, these days the whole "raise from" construct is nearly pointless. Used to be that exception chaining wasn't automatic though.
- aidos 5y ago
  I believe there’s a subtle difference in the meanings. You can raise or raise from. In either case the chaining means you get to see both messages. But one implies the original error is the cause. It’s so subtle that I don’t recall which is which!

Animats 5y ago

try: ... except Exception as e: Don't catch Exception at a low level. Catch OSError, which now includes EnvironmentError and IOError. This should cover all the things that can occur for external reasons. If you get some unexpected error such as a SyntaxError or TypeError, for example, that indicates a bug in the program. At the outermost level, catch Exception, and log a bug with a backtrace if an unhandled exception reaches that level. An exception is not an error code.

bedobi 5y ago

Some copy paste from previous posts Exception based error handling is so bad and unsafe that adopting functional error handling with Either, Try etc as implemented by functional addon libraries for many languages, while not yet common, in time it will become the new default even in OO languages. (just like it's been the default in functional languages for decades) Functional error handling types are much simpler, safer and more powerful. Simpler because they don't rely on dedicated syntax- they're just regular objects no different to any other object. Safer because unlike exceptions, they force callers to handle all potential outcomes, but no more. (no risk of ignoring errors and no risk of catching a higher level of error than desired, ubiquitous bugs in exception based error handling) Powerful because they support map, flatmap, applicative etc, making it easy to eg chain multiple computations together in desired ways, which is unwieldy and bug prone when using exceptions. What is wrong about dedicated syntax It adds complexity to the language! It could be that, when learning Java, Kotlin and any other language, we learn that methods return objects... and that's that. No weird dedicated syntax and magic, special treatment for returning anything other than the happy path, and the HUGE complexity that comes with it, eg the dedicated syntax itself and how it behaves, differences between checked and unchecked exceptions, hierarchies of exceptions etc etc. that makes our lives easier and our work more efficient But that's our point, it doesn't. Exceptions based error handling is unnecessary, hugely complex, doesn't compose at all, obfuscates or straight up hides what can go wrong with any given call, so leads to countless trivially preventable bugs... I could go on. And after decades of use, there's still no consensus about what exceptions should be or how they should be used. Exceptions are a failed experiment and I have no doubt that in ten years, Java, Kotlin and many other languages will acknowledge as much and move away from it the same way Joda Time outcompeted and replaced the horrible Java date and time library.

nesarkvechnep 5y ago
I've had great success with `Either` in Typescript. For each layer in the application I created ADTs representing the possible errors and used them in `Either`s. Each layer maps the lower layer errors to it's own.
nlitened 5y ago
From what I’ve seen so far, in reality people tend to overuse the `try!` macros, which leads to the same old bubbling exceptions semantic, but more verbose, less performant, and lacking stack traces.

gorpovitch 5y ago

When using a tool like Sentry, you might want to logger.exception(e) instead of a string error message, that way the whole stack trace with helpful debugging information (local variables...) is included in Sentry.

nicbou 5y ago
You might also want to use formatted strings too as it allows some logging tools to group the messages together.
tmarice 5y ago
logger.exception is basically a shortcut for logger.error(exc_info=True), so there's no need for explicitly assigning e as an error message.
- aidos 5y ago
  And if you look carefully you’ll find that doing logger.exception(e) will just cast e as a string, which is just the message in the e. So you end up just sending the message twice (along with the stack trace). Instead you can do logger.exception(“something helpful to give a little more context at a glance”)

mtlynch 5y ago

I disagree with most of this advice. I think Google's Python Style Guide provides exception guidance that leads to clearer code.[0] I agree with Animats's comment[1] about not catching the base Exception class, and the Google guidelines explicitly prohibit this, except at the outermost scope. One obvious flaw in catching Exception is if that if the user hits Ctrl+C to kill the application, but the code happens to be executing the ReceiptService at the time, the app will confusingly raise a ReceiptGenerationFailed exception instead of KeyboardInterrupt. (Edit: this isn't true, thanks to child comment for correction) The final code snippet shows four function calls within the same try even though the functions throw a disjoint set of exceptions. It's unclear to a reader at the callsite what exception handlers corrrelate with which functions. except OrderAlreadyInProgress as e: logger.info("Aborting emission request because it is already in progress!") return {"order_id": order_id, "order_status": e.order_status.value} To me, it feels sinful to use an exception to effectively "return" a value that the caller needs. I only use member fields in the exception to describe what went wrong. The original "bad" version of the code returned this information to the caller without raising an exception, and that felt more intuitive to me. else: return {"order_id": order_id, "order_status": order_status.value} I don't understand the purpose of the else at the end. I would just write the return within the try as: return {"order_id": order_id, "order_status": status_service.get_order_status(order_id) } [0] https://google.github.io/styleguide/pyguide.html https://google.github.io/styleguide/pyguide.html [1] https://news.ycombinator.com/item?id=27217301 https://news.ycombinator.com/item?id=27217301

tmarice 5y ago
https://docs.python.org/3/library/exceptions.html#KeyboardInterrupt https://docs.python.org/3/library/exceptions.html#KeyboardIn... `KeyboardInterrupt` inherits from `BaseException` instead of `Exception` exactly in order to avoid this situation.
- mtlynch 5y ago
  Oh, thanks for the correction. I remember running into this myself, but I must have done: except:
deleted 5y ago

nicbou 5y ago

One more thing: avoid broad catch clauses. Some exceptions ought to break things, like KeyboardInterrupt. By the way, you probably shouldn't wrap code lines on mobile. The code was difficult to read on mobile.