6 ms·

Python Language Summit: PEP 654 – Exception Groups and Except

91 points by BerislavLopac 5y ago

formerly_proven 5y ago
Recently I came up with a related idea for repeated error handling code, in the usual Python vein of introducing an abstraction to save 2-3 lines: try: foobar delegate handle_stuff: delegate handler_factory(something): except Foo: ... else: ... finally ... Where "delegate handle_stuff" means that for any exception handle_stuff is called with the exception tuple and the exception is considered handled if it returns truthy. "delegate" of course takes an expression so that you can have Proper Adult Fun Handling Exceptions.
- mfgs 5y ago
  How is that ultimately different from handling the exception under `except`?
  - coldtea 5y ago
    "in the usual Python vein of introducing an abstraction to save 2-3 lines"
    - ledauphin 5y ago
      this is what abstractions are for. they can be done poorly or well, but there's a reason iterators are favored in most use cases over for (i = 0, i < n; i++) { a[i] } - they're simpler to reason about because they restrict the conceptual operational space.
  - formerly_proven 5y ago
    The "delegate" statement encapsulates which exceptions are handled and how in the supplied callable expression. This is functionally different from repeating "except (FooExc, BarExc, BazExc) as exc: handle(exc)". It would also save two to three lines on average.
d0mine 5y ago
async with asyncio.TaskGroup() as g: g.create_task(concurrent_task1()) g.create_task(concurrent_task1()) reminds me of the structural concurrency: async with trio.open_nursery() as nursery: nursery.start_soon(myfunc) nursery.start_soon(anotherfunc) > nurseries ... rather a new control flow primitive that's just as fundamental as for loops or function calls https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g... > Implementing a better task spawning API in asyncio, inspired by Trio nurseries, was the main motivation for this PEP https://www.python.org/dev/peps/pep-0654/#motivation https://www.python.org/dev/peps/pep-0654/#motivation
- BiteCode_dev 5y ago
  It is explicitly inspired by it.
- meowface 5y ago
  Indeed, the screenshot at the bottom with that syntax says "inspired by Trio".
agumonkey 5y ago
IMO this is a worthy language agnostic topic, modeling and managing execution tree error cases has ever been an issue for my brain.
- _ix 5y ago
  I’m a bit of a pythonista, but I haven’t worked on anything very complex for a couple of years. Maybe my experience is still a little shallow in SWE topics, but can you reify this a bit? What kind of situations have you run into where handling multiple exceptions would have been handy? Maybe it’s telling that I’d have to admit that I’m not sure how big a difference there is between error code returns and exception handling.
  - agumonkey 5y ago
    It's just that the diagram shown in the article/blog were more inspiring than 99% what I heard about error abstractions on any language (and yeah it's not about python only). I just wished to read more about very general solutions on nested/stacked calls with exceptions and how to reduce the perimeter and make things more readable (honestly most of the time exception handling for me was logger.error or wrap/rethrow.. nothing exciting)
question000 5y ago
This is exactly the type of thing that should be in PEPs right now. It's practical, its forward thinking addressing an existing problem that's really prevalent in the py3 native async features and it's not incremental it addresses the problem in its totality.
Mathnerd314 5y ago
So this is replacing one cumbersome and unintuitive syntax (MultiError.catch) with a slightly shorter syntax (try except*). Given the amount of verbiage in the PEP and this blog post I'm guessing it will still be unintuitive though.
- luhn 5y ago
  The main advantage of this PEP over MultiError (IMO) is that standard except blocks will Just Work. For most use cases that will be sufficient, and those people will never have to think about this PEP.
- BiteCode_dev 5y ago
  Not just slightly shorter, because you need a handler for with MultiError.catch, which will be a function with and if and an instance check. Also, it doesn't give you easy "finally" and "else" clauses. And "except *" calls the same handler for single and multi-errors, so for most cases, you avoid code duplication. Plus, having a natural way to deal with this make the code easier to scan, not to forget IDE already have good support for a try/except, and wrapping things in it. It's 100% a win.
carapace 5y ago
This PEP is a footgun. Remember that exceptions are GOTO. If your exception-handling code is complex enough to be at all tricky you have a state machine and it behooves you to make that ad hoc machine explicit. Do not implement a multiplexing GOTO.
- kdmccormick 5y ago
  I see this sentiment a lot in language discussions; I really think the exceptions==goto parallel is overstated. Exceptions are a recognition of and abstraction over a common pattern from exception-less languages: "if <error state>, return early with a special value and some info on the error". Client functions follow a pattern of "if <error returned>, then handle the error or propogate it upwards". If you factor out that chain of early returns, you get "throw"/"raise". If you factor out the error handling, you get "catch"/"except". And if you turn that error info into an object/ADT, then you get the "Exception" class. Exceptions are not adding anything new; it's syntactic sugar over a more verbose pattern you'd probably be using anyway. You could totally implement a clunky but logically equivalent version of exceptions yourself in any language. Now, that being said, there's definitely a case to be made for avoiding over-abstracting and preferring explicitness to implicit magic. But that's much different than saying that exceptions are a under-abstraction like GOTO.
  - deleted 5y ago
    [deleted]
  - R0b0t1 5y ago
    I routinely encounter places where I am forced to use try..except where it would be better and more easily understood if I did not have to work around the fact I am forced to goto out of the current scope.
    - guenthert 5y ago
      I didn't downvote you, but I suspect you have been because you're mistaken. try...except doesn't force you to go out of scope, it keeps you in it. raising an exception leaves the scope, but nobody forces you to use it.
ab111111111 5y ago
Fantastic. An N dimensional error-handling space. That's exactly what Python is lacking.
- orf 5y ago
  It's definitely lacking it, but I guess this comment is sarcastic. So what do you think Python is lacking?
Recursing 5y ago
As mitsuhiko noted[0], I fear this might break a lot of existing generic exception handling code :( Am I wrong? Please someone tell me I'm wrong https://twitter.com/mitsuhiko/status/1364257824759504896 https://twitter.com/mitsuhiko/status/1364257824759504896
- BerislavLopac 5y ago
  I'm not sure what he meant by that comment. ExceptionGroup is an Exception subclass, so `except Exception` should work perfectly fine.

91 points by BerislavLopac 5y ago

Recently I came up with a related idea for repeated error handling code, in the usual Python vein of introducing an abstraction to save 2-3 lines: try: foobar delegate handle_stuff: delegate handler_factory(something): except Foo: ... else: ... finally ... Where "delegate handle_stuff" means that for any exception handle_stuff is called with the exception tuple and the exception is considered handled if it returns truthy. "delegate" of course takes an expression so that you can have Proper Adult Fun Handling Exceptions.

mfgs 5y ago
How is that ultimately different from handling the exception under `except`?
- coldtea 5y ago
  "in the usual Python vein of introducing an abstraction to save 2-3 lines"
  - ledauphin 5y ago
    this is what abstractions are for. they can be done poorly or well, but there's a reason iterators are favored in most use cases over for (i = 0, i < n; i++) { a[i] } - they're simpler to reason about because they restrict the conceptual operational space.
- formerly_proven 5y ago
  The "delegate" statement encapsulates which exceptions are handled and how in the supplied callable expression. This is functionally different from repeating "except (FooExc, BarExc, BazExc) as exc: handle(exc)". It would also save two to three lines on average.

d0mine 5y ago

async with asyncio.TaskGroup() as g: g.create_task(concurrent_task1()) g.create_task(concurrent_task1()) reminds me of the structural concurrency: async with trio.open_nursery() as nursery: nursery.start_soon(myfunc) nursery.start_soon(anotherfunc) > nurseries ... rather a new control flow primitive that's just as fundamental as for loops or function calls https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g... > Implementing a better task spawning API in asyncio, inspired by Trio nurseries, was the main motivation for this PEP https://www.python.org/dev/peps/pep-0654/#motivation https://www.python.org/dev/peps/pep-0654/#motivation

BiteCode_dev 5y ago
It is explicitly inspired by it.
meowface 5y ago
Indeed, the screenshot at the bottom with that syntax says "inspired by Trio".

agumonkey 5y ago

IMO this is a worthy language agnostic topic, modeling and managing execution tree error cases has ever been an issue for my brain.

_ix 5y ago
I’m a bit of a pythonista, but I haven’t worked on anything very complex for a couple of years. Maybe my experience is still a little shallow in SWE topics, but can you reify this a bit? What kind of situations have you run into where handling multiple exceptions would have been handy? Maybe it’s telling that I’d have to admit that I’m not sure how big a difference there is between error code returns and exception handling.
- agumonkey 5y ago
  It's just that the diagram shown in the article/blog were more inspiring than 99% what I heard about error abstractions on any language (and yeah it's not about python only). I just wished to read more about very general solutions on nested/stacked calls with exceptions and how to reduce the perimeter and make things more readable (honestly most of the time exception handling for me was logger.error or wrap/rethrow.. nothing exciting)

question000 5y ago

This is exactly the type of thing that should be in PEPs right now. It's practical, its forward thinking addressing an existing problem that's really prevalent in the py3 native async features and it's not incremental it addresses the problem in its totality.

Mathnerd314 5y ago

So this is replacing one cumbersome and unintuitive syntax (MultiError.catch) with a slightly shorter syntax (try except*). Given the amount of verbiage in the PEP and this blog post I'm guessing it will still be unintuitive though.

luhn 5y ago
The main advantage of this PEP over MultiError (IMO) is that standard except blocks will Just Work. For most use cases that will be sufficient, and those people will never have to think about this PEP.
BiteCode_dev 5y ago
Not just slightly shorter, because you need a handler for with MultiError.catch, which will be a function with and if and an instance check. Also, it doesn't give you easy "finally" and "else" clauses. And "except *" calls the same handler for single and multi-errors, so for most cases, you avoid code duplication. Plus, having a natural way to deal with this make the code easier to scan, not to forget IDE already have good support for a try/except, and wrapping things in it. It's 100% a win.

carapace 5y ago

This PEP is a footgun. Remember that exceptions are GOTO. If your exception-handling code is complex enough to be at all tricky you have a state machine and it behooves you to make that ad hoc machine explicit. Do not implement a multiplexing GOTO.

kdmccormick 5y ago
I see this sentiment a lot in language discussions; I really think the exceptions==goto parallel is overstated. Exceptions are a recognition of and abstraction over a common pattern from exception-less languages: "if <error state>, return early with a special value and some info on the error". Client functions follow a pattern of "if <error returned>, then handle the error or propogate it upwards". If you factor out that chain of early returns, you get "throw"/"raise". If you factor out the error handling, you get "catch"/"except". And if you turn that error info into an object/ADT, then you get the "Exception" class. Exceptions are not adding anything new; it's syntactic sugar over a more verbose pattern you'd probably be using anyway. You could totally implement a clunky but logically equivalent version of exceptions yourself in any language. Now, that being said, there's definitely a case to be made for avoiding over-abstracting and preferring explicitness to implicit magic. But that's much different than saying that exceptions are a under-abstraction like GOTO.
- deleted 5y ago
  [deleted]
- R0b0t1 5y ago
  I routinely encounter places where I am forced to use try..except where it would be better and more easily understood if I did not have to work around the fact I am forced to goto out of the current scope.
  - guenthert 5y ago
    I didn't downvote you, but I suspect you have been because you're mistaken. try...except doesn't force you to go out of scope, it keeps you in it. raising an exception leaves the scope, but nobody forces you to use it.

ab111111111 5y ago

Fantastic. An N dimensional error-handling space. That's exactly what Python is lacking.

orf 5y ago
It's definitely lacking it, but I guess this comment is sarcastic. So what do you think Python is lacking?

Recursing 5y ago

As mitsuhiko noted[0], I fear this might break a lot of existing generic exception handling code :( Am I wrong? Please someone tell me I'm wrong https://twitter.com/mitsuhiko/status/1364257824759504896 https://twitter.com/mitsuhiko/status/1364257824759504896

BerislavLopac 5y ago
I'm not sure what he meant by that comment. ExceptionGroup is an Exception subclass, so `except Exception` should work perfectly fine.