4 ms·

'with' gets better with python 3.1

22 points by init0 14y ago

famousactress 14y ago
This is cool.. But of the things that 3.1 has got me excited about, this doesn't really move the needle. It's pretty straight forward to throw together a context manager that deals with multiple files and access using `files.in` and `files.out` if you want to.
- masklinn 14y ago
  It's not that straightforward to create your own context manager, and creating a correct generic wrapper for the use case of composing multiple context managers is even harder (as demonstrated by the woes of contextlib.nested). And if you're going to create a custom "composed" context manager every single time, you're probably better off just nesting `with` statements.
pejoculant 14y ago
Compound with statements are nice, put this is not a 3.1 feature. This was added in 2.7: http://docs.python.org/reference/compound_stmts.html#the-with-statement http://docs.python.org/reference/compound_stmts.html#the-wit...
- koenigdavidmj 14y ago
  And you could hack this with contextlib.nested before.
- archangel_one 14y ago
  Despite the numbering, 2.7 was released a year after 3.1. It added a bunch of features that helped compatibility with 3.x.
  - throwaway54-762 14y ago
    Thanks, I forgot about that.
- briancurtin 14y ago
  It was a feature to both. Check the same document for the 3.1 release (and the What's New).
adharmad 14y ago
The syntax seems very similar to Lisp's with-open-file macro.
- adaszko 14y ago
  Since you can write your own "context managers," the with statement analogue from Lisp world would rather be unwind-protect.
- ufo 14y ago
  Yes, basically the only difference is that Python's with operates over a context management protocol involving some `__magic__` methods so it can work with other things in addition to files.
  - masklinn 14y ago
    You can also do that in common-lisp by using the special form `with-open-file` is built upon: unwind-protect[0]. `with-open-file` is just a macro over it, specialized for files handling. [0] http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/speope_unwind-protect.html http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...
sprash 14y ago
Since nothing got better with python 3.0 I'll stay with 2.x as long as I can.
- rflrob 14y ago
  Nothing got better with Python 3? What about the print function and true division? Sure, those are both there as from __future__ import's, but I'd definitely call those improvements. I haven't switched to Py3k yet either, but I'm just waiting for a few more packages I use heavily to get the bugs worked out.
  - archangel_one 14y ago
    Also, dict comprehensions. You can fake them in 2.x, but the syntax is ugly and cumbersome.
    - masklinn 14y ago
      dict comprehensions were backported to 2.7 $ python2.7 Python 2.7.3 (default, Apr 13 2012, 13:51:30) >>> {i: i+1 for i in range(5)} {0: 1, 1: 2, 2: 3, 3: 4, 4: 5}
  - masklinn 14y ago
    > What about the print function and true division? Those are actually fairly minor in the grand scheme of things (and compound `with` statements have been backported to 2.7). More interesting are: 3.0 * unicode strings by default and much better overall support of unicode * chained exceptions (I don't believe they've been backported) 3.2 * argparse merged to the stdlib * ElementTree 1.3 finalized in stdlib (3.2) * functools.lru_cache, itertools.accumulate (nothing game-changing, but nice to have in the stdlib) * threading.Barrier * concrete tzinfo implementation (for fixed UTC offsets) * contextlib.ContextDecorator (creates a type which works both as a contextmanager and a function decorator), and improved contextmanager support across the board (ftplib, popen, TarFile, ...) 3.3 (soon to be released) * Virtual environments shipped by default * Namespace packages * Kind-of obscure, but flexible string representations (strings can dynamically and transparently switch their implementation between latin-1, ucs-2 and ucs-4, this kills the narrow/wide compile-flag shit and generally makes strings more space-efficient) * sub-generator delegation, instead of: for item in sub_generator_function(): yield item you can just write: yield from sub_generator_function(). it also correctly forwards sent and thrown values (generator.send() and generator.throw()) to the sub-generator, so it actually translates into an unholy mess which now just works and allows more easily splitting a large function into sub-iterators. * lzma/xz support (7z) I'm actually excited for 3.3, there's plenty of cool stuff in there.
throwaway54-762 14y ago
Nevermind that this feature isn't particularly exciting... Python 3.1 is "old." Python 3.2 has been out for a while, 3.3 will be coming out soon... there are plenty of interesting features in 3.x. This is a small part of the overall picture. Rock on, Py3k devs.
AceJohnny2 14y ago
I don't get it. Can someone explain to me how this is different or better than: infile = open('site.log'); outfile = open('err.log', 'w') for line in infile: if 'ERROR' in line: outfile.write(line) infile.close(); outfile.close() Sure, you're saving a line with the files being automatically closed at the end of your scope, but hardly "MIND == BLOWN". What am I missing?
- Maascamp 14y ago
  Error handling.
- endlessvoid94 14y ago
  elegance?
- baddox 14y ago
  "MIND == BLOWN" is a hyperbole in almost every context, but the `with` statement is just cleaner code with fewer chances to screw up (e.g. by forgetting to close).
- deleted 14y ago
  [deleted]
- graue 14y ago
  You're missing that if an exception gets thrown, the files are still guaranteed to be closed as soon as you exit that 'with' block. Imagine you open 'infile', but then opening 'outfile' fails. To prevent a resource leak, now you need to close 'infile'. The linked code guarantees this. Likewise if there is any error which aborts the copying, both files will get closed and you don't have to worry so much about corner cases. See this for more: http://effbot.org/zone/python-with-statement.htm http://effbot.org/zone/python-with-statement.htm
- nnq 14y ago
  In a more complex application, an exception might be thrown before the last line that closes the file (and caught god knows where or not at all), so the files may not be closed... then again, if you actually try to catch the exceptions you get an extra level of indentation by using `with`. More... "philosophy" here: http://stackoverflow.com/questions/11373502/zen-of-python-vs-with-statement-philosophical-pondering http://stackoverflow.com/questions/11373502/zen-of-python-vs...
ef4 14y ago
I think "with" is a great example of where Python and Ruby differ philosophically. Python uses "with" to very explicitly implement their state manager protocol. Ruby doesn't need "with" because "with" is really just a limited special case of what you can do with blocks. The Python fans will counter that their less general solution is harder to abuse.
- masklinn 14y ago
  > The Python fans will counter that their less general solution is harder to abuse. Not sure why anybody would say that, you can plenty abuse `with` if you want to have fun, the only thing you can't do is "dual-use" the block really (grab a context and invoke the block multiple times). And that you could always do with a decorator, if you're going there. The only thing `with` is is less flexible/general. `unwind-protect` or `BlockClosure#ensure:` are probably much better examples of the philosophy opposite python though, I've always found ruby rather half-assed.
jlarocco 14y ago
Not being mean, but this article kinda made me laugh. Python 3.1 has been out over 3 years[1]. And with the over the top "Mind == BLOWN"... Somebody should have told him about that release 3 years ago :-) http://www.python.org/doc/versions/ http://www.python.org/doc/versions/

22 points by init0 14y ago

This is cool.. But of the things that 3.1 has got me excited about, this doesn't really move the needle. It's pretty straight forward to throw together a context manager that deals with multiple files and access using `files.in` and `files.out` if you want to.

masklinn 14y ago
It's not that straightforward to create your own context manager, and creating a correct generic wrapper for the use case of composing multiple context managers is even harder (as demonstrated by the woes of contextlib.nested). And if you're going to create a custom "composed" context manager every single time, you're probably better off just nesting `with` statements.

pejoculant 14y ago

Compound with statements are nice, put this is not a 3.1 feature. This was added in 2.7: http://docs.python.org/reference/compound_stmts.html#the-with-statement http://docs.python.org/reference/compound_stmts.html#the-wit...

koenigdavidmj 14y ago
And you could hack this with contextlib.nested before.
archangel_one 14y ago
Despite the numbering, 2.7 was released a year after 3.1. It added a bunch of features that helped compatibility with 3.x.
- throwaway54-762 14y ago
  Thanks, I forgot about that.
briancurtin 14y ago
It was a feature to both. Check the same document for the 3.1 release (and the What's New).

adharmad 14y ago

The syntax seems very similar to Lisp's with-open-file macro.

adaszko 14y ago
Since you can write your own "context managers," the with statement analogue from Lisp world would rather be unwind-protect.
ufo 14y ago
Yes, basically the only difference is that Python's with operates over a context management protocol involving some `__magic__` methods so it can work with other things in addition to files.
- masklinn 14y ago
  You can also do that in common-lisp by using the special form `with-open-file` is built upon: unwind-protect[0]. `with-open-file` is just a macro over it, specialized for files handling. [0] http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/speope_unwind-protect.html http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...

sprash 14y ago

Since nothing got better with python 3.0 I'll stay with 2.x as long as I can.

rflrob 14y ago
Nothing got better with Python 3? What about the print function and true division? Sure, those are both there as from __future__ import's, but I'd definitely call those improvements. I haven't switched to Py3k yet either, but I'm just waiting for a few more packages I use heavily to get the bugs worked out.
- archangel_one 14y ago
  Also, dict comprehensions. You can fake them in 2.x, but the syntax is ugly and cumbersome.
  - masklinn 14y ago
    dict comprehensions were backported to 2.7 $ python2.7 Python 2.7.3 (default, Apr 13 2012, 13:51:30) >>> {i: i+1 for i in range(5)} {0: 1, 1: 2, 2: 3, 3: 4, 4: 5}
- masklinn 14y ago
  > What about the print function and true division? Those are actually fairly minor in the grand scheme of things (and compound `with` statements have been backported to 2.7). More interesting are: 3.0 * unicode strings by default and much better overall support of unicode * chained exceptions (I don't believe they've been backported) 3.2 * argparse merged to the stdlib * ElementTree 1.3 finalized in stdlib (3.2) * functools.lru_cache, itertools.accumulate (nothing game-changing, but nice to have in the stdlib) * threading.Barrier * concrete tzinfo implementation (for fixed UTC offsets) * contextlib.ContextDecorator (creates a type which works both as a contextmanager and a function decorator), and improved contextmanager support across the board (ftplib, popen, TarFile, ...) 3.3 (soon to be released) * Virtual environments shipped by default * Namespace packages * Kind-of obscure, but flexible string representations (strings can dynamically and transparently switch their implementation between latin-1, ucs-2 and ucs-4, this kills the narrow/wide compile-flag shit and generally makes strings more space-efficient) * sub-generator delegation, instead of: for item in sub_generator_function(): yield item you can just write: yield from sub_generator_function(). it also correctly forwards sent and thrown values (generator.send() and generator.throw()) to the sub-generator, so it actually translates into an unholy mess which now just works and allows more easily splitting a large function into sub-iterators. * lzma/xz support (7z) I'm actually excited for 3.3, there's plenty of cool stuff in there.

throwaway54-762 14y ago

Nevermind that this feature isn't particularly exciting... Python 3.1 is "old." Python 3.2 has been out for a while, 3.3 will be coming out soon... there are plenty of interesting features in 3.x. This is a small part of the overall picture. Rock on, Py3k devs.

AceJohnny2 14y ago

I don't get it. Can someone explain to me how this is different or better than: infile = open('site.log'); outfile = open('err.log', 'w') for line in infile: if 'ERROR' in line: outfile.write(line) infile.close(); outfile.close() Sure, you're saving a line with the files being automatically closed at the end of your scope, but hardly "MIND == BLOWN". What am I missing?

Maascamp 14y ago
Error handling.
endlessvoid94 14y ago
elegance?
baddox 14y ago
"MIND == BLOWN" is a hyperbole in almost every context, but the `with` statement is just cleaner code with fewer chances to screw up (e.g. by forgetting to close).
deleted 14y ago
[deleted]
graue 14y ago
You're missing that if an exception gets thrown, the files are still guaranteed to be closed as soon as you exit that 'with' block. Imagine you open 'infile', but then opening 'outfile' fails. To prevent a resource leak, now you need to close 'infile'. The linked code guarantees this. Likewise if there is any error which aborts the copying, both files will get closed and you don't have to worry so much about corner cases. See this for more: http://effbot.org/zone/python-with-statement.htm http://effbot.org/zone/python-with-statement.htm
nnq 14y ago
In a more complex application, an exception might be thrown before the last line that closes the file (and caught god knows where or not at all), so the files may not be closed... then again, if you actually try to catch the exceptions you get an extra level of indentation by using `with`. More... "philosophy" here: http://stackoverflow.com/questions/11373502/zen-of-python-vs-with-statement-philosophical-pondering http://stackoverflow.com/questions/11373502/zen-of-python-vs...

ef4 14y ago

I think "with" is a great example of where Python and Ruby differ philosophically. Python uses "with" to very explicitly implement their state manager protocol. Ruby doesn't need "with" because "with" is really just a limited special case of what you can do with blocks. The Python fans will counter that their less general solution is harder to abuse.

masklinn 14y ago
> The Python fans will counter that their less general solution is harder to abuse. Not sure why anybody would say that, you can plenty abuse `with` if you want to have fun, the only thing you can't do is "dual-use" the block really (grab a context and invoke the block multiple times). And that you could always do with a decorator, if you're going there. The only thing `with` is is less flexible/general. `unwind-protect` or `BlockClosure#ensure:` are probably much better examples of the philosophy opposite python though, I've always found ruby rather half-assed.

jlarocco 14y ago

Not being mean, but this article kinda made me laugh. Python 3.1 has been out over 3 years[1]. And with the over the top "Mind == BLOWN"... Somebody should have told him about that release 3 years ago :-) http://www.python.org/doc/versions/ http://www.python.org/doc/versions/