5 ms·
Though I very much like that this exists, and the documentation for contextlib describes its need and use very well[1], I do want to point out that this has alw
by Groxx 6y ago
Though I very much like that this exists, and the documentation for contextlib describes its need and use very well[1], I do want to point out that this has always been possible with a tiny bit of recursive glue code like this[2], which works fine even in python 2:
@contextmanager
def multi(*cms):
if len(cms) > 0:
with cms[0]:
with multi(*cms[1:]):
yield
else:
yield
# use with 0..N items, it behaves the same
with multi(...):
pass
Prior to finding ExitStack, I've frequently seen people either give up and do something obviously wrong, or try to manually manage their own stack of __enter__ and __exit__ funcs (which works, but exception handling is non-obvious and often loses useful info). Personally I prefer the recursive route, since it very obviously behaves the same as if you wrote the nesting out by hand. It even retains accurate stack traces "naturally", unlike most non-ExitStack tactics I've seen through the years. ExitStack's stack-correcting tactic is... a bit more "interesting"[3], though AFAIK it achieves roughly the same stack trace in the end.
Since ExitStack exists, and is likely more efficient (by how much, I have no idea), it's probably a better choice for most situations. And it's just broadly a bit nicer for things that yield resources.
It's always worth browsing stdlib documentation to find gems like this, there are quite a few neat things in most languages that seem fairly unknown.
[1]: https://docs.python.org/3/library/contextlib.html#supporting-a-variable-number-of-context-managers https://docs.python.org/3/library/contextlib.html#supporting...
[2]: https://repl.it/@Groxx/TepidRosybrownLocatorprogram#main.py https://repl.it/@Groxx/TepidRosybrownLocatorprogram#main.py
[3]: https://github.com/python/cpython/blob/master/Lib/contextlib.py#L564 https://github.com/python/cpython/blob/master/Lib/contextlib...
- hchasestevens 6y agoI always thought it was a shame that `nested`[0] (the stdlib equivalent to your `multi` above) was deprecated in 2.7 and, ultimately, removed in favor of `ExitStack` -- despite its flaws, I think it was more beginner-friendly. [0]: https://docs.python.org/2.7/library/contextlib.html#contextlib.nested https://docs.python.org/2.7/library/contextlib.html#contextl...
- Groxx 6y agoYeah, and the source for `nested` is a lot easier to follow than ExitStack. Conceptually it's the same as mine, but with a loop rather than recursion, and it'll yield values (not hard to do with a recursive one, I just didn't feel like it): https://github.com/python/cpython/blob/2.7/Lib/contextlib.py#L89 https://github.com/python/cpython/blob/2.7/Lib/contextlib.py... Seems like the __new__/__init__ issues exist with ExitStack as well, unlike with `with a, b, c`? So it's not really resolved / improved behaviorally, though ExitStack does have a few new uses (e.g. managing non-context-manager resources).
- andreareina 6y agoHaven't used it since I'm just hearing about it now, but seems like the big win to using ExitStack is that you're not required to acquire all the resources at the same time. So you can acquire one resource, do some stuff, then acquire another resource based on something that is dependent on the first without more indentation.
- freininghaus 6y agoI agree that the recursive solution is nice and simple. We've also used it at work. However, you can run into "maximum recursion depth exceeded" issues if the number of context managers is extremely large. ExitStack handles this situation nicely [1]. Note that I'm not saying that using so many context managers is a good idea ;-) In our case, using ExitStack (which also has a backport to Python 2 [2]) was the simplest fix though. [1] https://github.com/freininghaus/notes/blob/master/misc-notebooks/2020-11-11-python-exitstack-vs-recursion.ipynb https://github.com/freininghaus/notes/blob/master/misc-noteb... or https://nbviewer.jupyter.org/github/freininghaus/notes/blob/master/misc-notebooks/2020-11-11-python-exitstack-vs-recursion.ipynb https://nbviewer.jupyter.org/github/freininghaus/notes/blob/... [2] https://contextlib2.readthedocs.io/en/stable/#contextlib2.ExitStack https://contextlib2.readthedocs.io/en/stable/#contextlib2.Ex...