3 ms·

Is it non-idiomatic to do "for [line] in [file object]"? I use "for line in open('file')" all the time, for similar reasons as presented in the article. "for li

Is it non-idiomatic to do "for [line] in [file object]"? I use "for line in open('file')" all the time, for similar reasons as presented in the article. "for line in sys.stdin" is basically the same pattern, just with a different file object. Edit: The idiom's mentioned in the Python docs on IO [1] as "memory efficient, fast, and leads to simple code" [1] http://docs.python.org/2/tutorial/inputoutput.html#methods-of-file-objects http://docs.python.org/2/tutorial/inputoutput.html#methods-o...

icebraining 13y ago
Nitpick: it's better to do "with open('file') as file: for line in file: ..." instead, but otherwise yes, iterating over file objects is great. Another option is using mmap[1], particularly when the file is already in memory or you need more random access to it. It worked well when I was trying to parse some lines from the end of an open log file. [1] http://docs.python.org/2/library/mmap.html http://docs.python.org/2/library/mmap.html
- richardjs 13y ago
  Is there a difference between the two? I assumed that without the "with", the file would still be closed once the loop was exited (and thus the reference to the file is dropped), but I'm open to the possibility that I'm mistaken.
  - tjgq 13y ago
    Without the context manager (with statement), the underlying file is closed when the file object is garbage-collected. In CPython, since reference counting is used for GC, this occurs when the loop exits. However, other implementations (e.g. PyPy) may use different schemes that do not guarantee collection as soon as objects go out of scope. As an extreme, a valid and occasionally useful GC strategy is to never collect anything at all [0]! Hence, if you want to portably ensure the file is closed, you should either use the context manager or call close() explicitly. [0] http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047...
  - ijl 13y ago
    Relying on GC to close the file isn't safe. See http://stackoverflow.com/questions/1834556/does-a-file-object-automatically-close-when-its-reference-count-hits-zero http://stackoverflow.com/questions/1834556/does-a-file-objec...
deleted 13y ago
[deleted]
deleted 13y ago
[deleted]

icebraining 13y ago

Nitpick: it's better to do "with open('file') as file: for line in file: ..." instead, but otherwise yes, iterating over file objects is great. Another option is using mmap[1], particularly when the file is already in memory or you need more random access to it. It worked well when I was trying to parse some lines from the end of an open log file. [1] http://docs.python.org/2/library/mmap.html http://docs.python.org/2/library/mmap.html

richardjs 13y ago
Is there a difference between the two? I assumed that without the "with", the file would still be closed once the loop was exited (and thus the reference to the file is dropped), but I'm open to the possibility that I'm mistaken.
- tjgq 13y ago
  Without the context manager (with statement), the underlying file is closed when the file object is garbage-collected. In CPython, since reference counting is used for GC, this occurs when the loop exits. However, other implementations (e.g. PyPy) may use different schemes that do not guarantee collection as soon as objects go out of scope. As an extreme, a valid and occasionally useful GC strategy is to never collect anything at all [0]! Hence, if you want to portably ensure the file is closed, you should either use the context manager or call close() explicitly. [0] http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047...
- ijl 13y ago
  Relying on GC to close the file isn't safe. See http://stackoverflow.com/questions/1834556/does-a-file-object-automatically-close-when-its-reference-count-hits-zero http://stackoverflow.com/questions/1834556/does-a-file-objec...

deleted 13y ago

[deleted]

deleted 13y ago

[deleted]