6 ms·

> What I like about the Rust (and to a lesser extent, Go) approach to error handling is, you know up front if and what errors a function might return. And addit

> What I like about the Rust (and to a lesser extent, Go) approach to error handling is, you know up front if and what errors a function might return. And additionally, the compiler forces you to at least consider how you’re going to deal with it. > By comparison, the python approach to dealing with exceptions is quite chunky > try: > function() > except SomeException: > # etc > I think this, in fact, encourages just letting exceptions bubble up, rather than fussing about with handling them. Letting exceptions bubble up is usually the right thing to do. Handle them centrally, e.g. in middleware for a web application.

mindwok 3y ago
Agree. I spent quite a lot of effort at a previous role trying to convince junior developers to stop using except pretty much everywhere and just let the program explode, because that's really what you want most of the time. The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. In that case, it's very useful to know ahead of time what errors might be returned.
- dotancohen 3y ago
  > The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. Almost every time I've heard this argument, I've disagreed with it. I think that the only time using Exceptions for flow control was valid was some string-validation technique (like ensuring that a string was JSON or an email address) that could only throw an exception on error. I'd love to see _your_ use case, though.
  - stevesimmons 3y ago
    > The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. Using exceptions for control flow is, imo, fine within a single function (e.g. for x in y: try: blah except KeyError: continue). Using exceptions for control flow across function boundaries seems a recipe for confusion, unless very clearly documented.
  - tcfhgj 3y ago
    The async library e.g. uses exceptions for queues.
    - bombolo 3y ago
      ?
      - tcfhgj 3y ago
        What is not clear?
        bombolo 3y ago
        Can you make an example of what you mean?
  - skitter 3y ago
    I'm confused — how would one use exceptions if not for control flow?
    - tialaramex 3y ago
      In general exceptions are error handling, for "exceptional" situations. For example suppose we're supposed to ensure the water flow is at least X litres per second, but now X is NaN, that's nonsense, raise an exception. You are correct that unlike a Result sum type, or an error code, or numerous other error solutions the exceptions inherently also change control flow. In Python though it's usual to (ab)use exceptions for other flows, for example to indicate that we're done, lets just throw a Done exception. The distinction is why Rust reifies control flow as the ControlFlow type, https://doc.rust-lang.org/core/ops/enum.ControlFlow.html https://doc.rust-lang.org/core/ops/enum.ControlFlow.html -- so we have shared vocabulary to distinguish "This failed w/ error X" Result::Err(X) from "We should stop now, result Y" ControlFlow::Break(Y) even though sometimes it's true that we're stopping because of an error so we end up transforming Result::Err(X) into ControlFlow::Break(X). Doing that transformation is what the Try trait, the operator ? does today it calls Try::branch(result) and if that's ControlFlow::Break(X) your function exits with X, if it's ControlFlow::Continue(C) then the operator's result is C.
  - Skeime 3y ago
    I think any type of parsing benefits from some happy-path/unhappy-path support in the language, because parsing essentially is about figuring out whether your input conforms to some format and then translating it into a more useful representation. And it often works in a nested way, where you parse sub-parts of your input and then combine those parts later. If you don’t use some form of exceptions/result types/CPS, you end up with lots of code like if first part of input_data has format a: extract first part of output from input_data You need the checks because otherwise your program will blow up in the extract step, which in parsing is often not what you want, as you might need to try a different way of parsing the same thing first. Internally, the check and the extraction often do pretty similar things, so you’re duplicating your work. Here, exceptions for control flow can help, because instead of the check, you just try to extract, and if it doesn’t work, you try the next version. (However, result types usually work better for this, in programming languages that support them.) And as the last point, in my experience, many programming problems are handled cleanest when treating them as the problem of parsing some form of input (not necessarily a string) into some form of output. So this comes up more often than one might first think. [Also, Python uses exceptions for some types of control flow internally. For example, StopIteration is thrown by Iterators for the (rather unexceptional) case that the Iterator is finished producing output. Admittedly, you can almost always just use a for loop, except when you want access to the StopIteration’s value at the end.]
    - fbdab103 3y ago
      Checking if a string represents an integer. There is a great SO post[0] with various alternatives, which all feel worse. [0] https://stackoverflow.com/questions/1265665/how-can-i-check-if-a-string-represents-an-int-without-using-try-except https://stackoverflow.com/questions/1265665/how-can-i-check-...
  - Scarblac 3y ago
    The way that an iterator signals that it is done iterating is by raising StopIteration ( https://docs.python.org/3/library/exceptions.html#StopIteration https://docs.python.org/3/library/exceptions.html#StopIterat... ). I don't know if it's still the case in Python 3, but in 2 that exception was core to how for loops and list comprehensions worked.
    - ok123456 3y ago
      This is still the case.
IshKebab 3y ago
With Rust & Go style error handling you generally still do bubble up errors and handle them centrally. But you don't always want to do that. Sometimes you want to handle errors locally and e.g. retry things, or try something else in case of an error or ignore some errors etc. You also should add context (`raise ConfigParseError() from int_parse_error`) which approximately 0% of people do with exceptions. Handling errors locally is difficult to do with exception handling because it's impossible to tell what errors a function might produce (if any!). It's also more verbose (especially compared to Rust) which just makes it annoying. Checked exceptions are the usual answer to this but they're so tedious most people don't bother with them. C++ even dropped support for them from the language. Rust's error handling is much better than exceptions in almost every way.
- riffraff 3y ago
  Why do you think nobody wraps exceptions? Everyone did in every project in every language I've worked on. Every rest API client library will wrap the http library errors which will wrap network errors. Languages even directly support the notion of keeping a reference to the original exception. If anything, people do it too much.
  - berkes 3y ago
    There's always that edge case, say "FileReadTimedOut" presented as a UncaughtServerError that no developer ever knew the existence of, let alone thought to wrap it (in, say, the LogoStorage). With Rust this situation is impossible. Unexpected errors are made impossible by the type checker, a dev must handle it. Sure, a dev may say "meh, I'll just .into() (or wrap, if you want) any error that file-io gives into NotFound" but she does so knowingly and actively.
diarrhea 3y ago
Handle errors as soon as you can is my stance. If a low-level function raises, but its immediate caller knows how to handle that specific exception, handle it (for example, turn a zero division error into None, if it makes sense for the domain). Only bubble up if your handling consists of reraising anyway and no business logic can be applied (which might be the majority of cases).
- jdeaton 3y ago
  This is interestingly divergent from the advice given in “Philosophy of Software Design” by John O. which is “exceptions are most useful when they're thrown the furthest”
  - Scarblac 3y ago
    Note the comment you reply adds "this might be the majority of cases". There are exceptions that are a normal result of some function call, and they can be handled by the caller. But most exceptions are of the kind that you want to pass to some general error reporting thing far away.
  - diarrhea 3y ago
    Funny you’re mentioning that exact book. I almost put it as the source in my original comment, but maybe I remember wrong. What I remember is the “deep interfaces” advice, which is great. It encourages small surfaces but deep implementations. Hiding the potentially large list of raisable exceptions helps in keeping an interface or API small. Errors are handled transparently and the user only needs to deal with those they know how to handle. What good is it to raise a deep error that by the time it reaches the top levels (like a web servers middleware loop), it can no longer be handled reasonably and has to be thrown? That’s why, if you can handle it, do so ASAP. That’s what I remember from that book. Again, perhaps I’m misremembering.
  - MereInterest 3y ago
    I don’t think the pieces of advice contradict each other. If exceptions are handled at the lowest scope possible, and exceptions are most useful when the rise above many scopes, then exceptions are most useful when the scope that encounters the error is very deep compared to the scope that can handle the error. This agrees with my intuition, that error codes are most fragile when they must be checked and re-raised across many scopes.
ramraj07 3y ago
A good fraction of bugs and exceptions I’ve seen in real python code bases is because the code itself didn’t handle the None edge cases at the exact moment they get created, and hence fail in a different place and let the debugging engineer scratching their head. If you write typed python code that lets you avoid this while writing the bad code at the beginning itself, that’s a win. Actual runtime errors caused due to invalid user input should of course throw appropriate exceptions deep in the code and bubble up and be handled in middleware.
the_gipsy 3y ago
Usually? Yea there is a lot of stuff that you handle "centrally", but there is also a lot of stuff that you should handle locally. E.g. "try this then that". Or stuff that you try, just log or save the error, and continue. Exceptions are obviously great for bubbling up, but just awful for handling locally. I don't know what it is: maybe it's the horrible try/catch control flow that breaks everything up, or the fact that all functions can "return" exceptions apart from the ones in the signature (if any), so you're never quite sure what happens or what to do.
duckmysick 3y ago
> Handle them centrally, e.g. in middleware for a web application. So, for a Django app is it better to use their built-in middleware framework or something third-party that sits outside of Django? https://docs.djangoproject.com/en/4.2/topics/http/middleware/ https://docs.djangoproject.com/en/4.2/topics/http/middleware...
- robertlagrant 3y ago
  Depends on what you mean by "sits outside of". If an exception reaches the response code in a web application, it's going to be converted to an HTTP error code and sent. You want to be thinking about what you want your API (or website) to expose to the user and work back from there.
BerislavLopac 3y ago
I like to say that the whole point of exceptions is not to handle them -- except when you know exactly which exception can happen and how to handle it. Everything else should be allowed to bubble up and handle on the highest level, whether it's by returning an error HTTP code, showing an error message to the user, or simply logging it at the right level. This is why the try: block should never contain more than one single line, with the except block(s) limited only to the errors that can be expected from that line and that you explicitly want to handle at that point. Trying multiple lines and handling generic levels is a perfect way to get errors that are difficult to track.

mindwok 3y ago

Agree. I spent quite a lot of effort at a previous role trying to convince junior developers to stop using except pretty much everywhere and just let the program explode, because that's really what you want most of the time. The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. In that case, it's very useful to know ahead of time what errors might be returned.

dotancohen 3y ago
> The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. Almost every time I've heard this argument, I've disagreed with it. I think that the only time using Exceptions for flow control was valid was some string-validation technique (like ensuring that a string was JSON or an email address) that could only throw an exception on error. I'd love to see _your_ use case, though.
- stevesimmons 3y ago
  > The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. Using exceptions for control flow is, imo, fine within a single function (e.g. for x in y: try: blah except KeyError: continue). Using exceptions for control flow across function boundaries seems a recipe for confusion, unless very clearly documented.
- tcfhgj 3y ago
  The async library e.g. uses exceptions for queues.
  - bombolo 3y ago
    ?
    - tcfhgj 3y ago
      What is not clear?
      - bombolo 3y ago
        Can you make an example of what you mean?
- skitter 3y ago
  I'm confused — how would one use exceptions if not for control flow?
  - tialaramex 3y ago
    In general exceptions are error handling, for "exceptional" situations. For example suppose we're supposed to ensure the water flow is at least X litres per second, but now X is NaN, that's nonsense, raise an exception. You are correct that unlike a Result sum type, or an error code, or numerous other error solutions the exceptions inherently also change control flow. In Python though it's usual to (ab)use exceptions for other flows, for example to indicate that we're done, lets just throw a Done exception. The distinction is why Rust reifies control flow as the ControlFlow type, https://doc.rust-lang.org/core/ops/enum.ControlFlow.html https://doc.rust-lang.org/core/ops/enum.ControlFlow.html -- so we have shared vocabulary to distinguish "This failed w/ error X" Result::Err(X) from "We should stop now, result Y" ControlFlow::Break(Y) even though sometimes it's true that we're stopping because of an error so we end up transforming Result::Err(X) into ControlFlow::Break(X). Doing that transformation is what the Try trait, the operator ? does today it calls Try::branch(result) and if that's ControlFlow::Break(X) your function exits with X, if it's ControlFlow::Continue(C) then the operator's result is C.
- Skeime 3y ago
  I think any type of parsing benefits from some happy-path/unhappy-path support in the language, because parsing essentially is about figuring out whether your input conforms to some format and then translating it into a more useful representation. And it often works in a nested way, where you parse sub-parts of your input and then combine those parts later. If you don’t use some form of exceptions/result types/CPS, you end up with lots of code like if first part of input_data has format a: extract first part of output from input_data You need the checks because otherwise your program will blow up in the extract step, which in parsing is often not what you want, as you might need to try a different way of parsing the same thing first. Internally, the check and the extraction often do pretty similar things, so you’re duplicating your work. Here, exceptions for control flow can help, because instead of the check, you just try to extract, and if it doesn’t work, you try the next version. (However, result types usually work better for this, in programming languages that support them.) And as the last point, in my experience, many programming problems are handled cleanest when treating them as the problem of parsing some form of input (not necessarily a string) into some form of output. So this comes up more often than one might first think. [Also, Python uses exceptions for some types of control flow internally. For example, StopIteration is thrown by Iterators for the (rather unexceptional) case that the Iterator is finished producing output. Admittedly, you can almost always just use a for loop, except when you want access to the StopIteration’s value at the end.]
  - fbdab103 3y ago
    Checking if a string represents an integer. There is a great SO post[0] with various alternatives, which all feel worse. [0] https://stackoverflow.com/questions/1265665/how-can-i-check-if-a-string-represents-an-int-without-using-try-except https://stackoverflow.com/questions/1265665/how-can-i-check-...
- Scarblac 3y ago
  The way that an iterator signals that it is done iterating is by raising StopIteration ( https://docs.python.org/3/library/exceptions.html#StopIteration https://docs.python.org/3/library/exceptions.html#StopIterat... ). I don't know if it's still the case in Python 3, but in 2 that exception was core to how for loops and list comprehensions worked.
  - ok123456 3y ago
    This is still the case.

IshKebab 3y ago

With Rust & Go style error handling you generally still do bubble up errors and handle them centrally. But you don't always want to do that. Sometimes you want to handle errors locally and e.g. retry things, or try something else in case of an error or ignore some errors etc. You also should add context (`raise ConfigParseError() from int_parse_error`) which approximately 0% of people do with exceptions. Handling errors locally is difficult to do with exception handling because it's impossible to tell what errors a function might produce (if any!). It's also more verbose (especially compared to Rust) which just makes it annoying. Checked exceptions are the usual answer to this but they're so tedious most people don't bother with them. C++ even dropped support for them from the language. Rust's error handling is much better than exceptions in almost every way.

riffraff 3y ago
Why do you think nobody wraps exceptions? Everyone did in every project in every language I've worked on. Every rest API client library will wrap the http library errors which will wrap network errors. Languages even directly support the notion of keeping a reference to the original exception. If anything, people do it too much.
- berkes 3y ago
  There's always that edge case, say "FileReadTimedOut" presented as a UncaughtServerError that no developer ever knew the existence of, let alone thought to wrap it (in, say, the LogoStorage). With Rust this situation is impossible. Unexpected errors are made impossible by the type checker, a dev must handle it. Sure, a dev may say "meh, I'll just .into() (or wrap, if you want) any error that file-io gives into NotFound" but she does so knowingly and actively.

diarrhea 3y ago

Handle errors as soon as you can is my stance. If a low-level function raises, but its immediate caller knows how to handle that specific exception, handle it (for example, turn a zero division error into None, if it makes sense for the domain). Only bubble up if your handling consists of reraising anyway and no business logic can be applied (which might be the majority of cases).

jdeaton 3y ago
This is interestingly divergent from the advice given in “Philosophy of Software Design” by John O. which is “exceptions are most useful when they're thrown the furthest”
- Scarblac 3y ago
  Note the comment you reply adds "this might be the majority of cases". There are exceptions that are a normal result of some function call, and they can be handled by the caller. But most exceptions are of the kind that you want to pass to some general error reporting thing far away.
- diarrhea 3y ago
  Funny you’re mentioning that exact book. I almost put it as the source in my original comment, but maybe I remember wrong. What I remember is the “deep interfaces” advice, which is great. It encourages small surfaces but deep implementations. Hiding the potentially large list of raisable exceptions helps in keeping an interface or API small. Errors are handled transparently and the user only needs to deal with those they know how to handle. What good is it to raise a deep error that by the time it reaches the top levels (like a web servers middleware loop), it can no longer be handled reasonably and has to be thrown? That’s why, if you can handle it, do so ASAP. That’s what I remember from that book. Again, perhaps I’m misremembering.
- MereInterest 3y ago
  I don’t think the pieces of advice contradict each other. If exceptions are handled at the lowest scope possible, and exceptions are most useful when the rise above many scopes, then exceptions are most useful when the scope that encounters the error is very deep compared to the scope that can handle the error. This agrees with my intuition, that error codes are most fragile when they must be checked and re-raised across many scopes.

ramraj07 3y ago

A good fraction of bugs and exceptions I’ve seen in real python code bases is because the code itself didn’t handle the None edge cases at the exact moment they get created, and hence fail in a different place and let the debugging engineer scratching their head. If you write typed python code that lets you avoid this while writing the bad code at the beginning itself, that’s a win. Actual runtime errors caused due to invalid user input should of course throw appropriate exceptions deep in the code and bubble up and be handled in middleware.

the_gipsy 3y ago

Usually? Yea there is a lot of stuff that you handle "centrally", but there is also a lot of stuff that you should handle locally. E.g. "try this then that". Or stuff that you try, just log or save the error, and continue. Exceptions are obviously great for bubbling up, but just awful for handling locally. I don't know what it is: maybe it's the horrible try/catch control flow that breaks everything up, or the fact that all functions can "return" exceptions apart from the ones in the signature (if any), so you're never quite sure what happens or what to do.

duckmysick 3y ago

> Handle them centrally, e.g. in middleware for a web application. So, for a Django app is it better to use their built-in middleware framework or something third-party that sits outside of Django? https://docs.djangoproject.com/en/4.2/topics/http/middleware/ https://docs.djangoproject.com/en/4.2/topics/http/middleware...

robertlagrant 3y ago
Depends on what you mean by "sits outside of". If an exception reaches the response code in a web application, it's going to be converted to an HTTP error code and sent. You want to be thinking about what you want your API (or website) to expose to the user and work back from there.

BerislavLopac 3y ago

I like to say that the whole point of exceptions is not to handle them -- except when you know exactly which exception can happen and how to handle it. Everything else should be allowed to bubble up and handle on the highest level, whether it's by returning an error HTTP code, showing an error message to the user, or simply logging it at the right level. This is why the try: block should never contain more than one single line, with the except block(s) limited only to the errors that can be expected from that line and that you explicitly want to handle at that point. Trying multiple lines and handling generic levels is a perfect way to get errors that are difficult to track.