3 ms·
> - reference counting is an implementation detail, it is not part of the spec > - that's because real implementations exist where refcounting does not exist (
by masklinn 6y ago
> - reference counting is an implementation detail, it is not part of the spec
> - that's because real implementations exist where refcounting does not exist (ironpython, pypy, jython)
Exactly. That was a major if not the major drive for context managers & properly closing files: the PFS and the language team were getting much more supportive of third-party implementations, and much more aware of issues in "normal" Python code e.g. though it didn't work out well in the end, ctypes also got merged into the stdlib in 2.5 with the express purpose of limiting the need for writing cpython extensions to bind to native libraries.
> - the random system resource will eventually be garbage collected
Not necessarily! In fact this is one of the major problems for non-memory resources in GC'd languages: the GC is triggered by memory pressure and that's it. Because it knows nothing about fd pressure, if fd consumption is not matched by enough memory consumption (or a properly matching pattern thereof) it's rather possible to keep leaking fds without the GC ever running (or ever running a collection which reclaims your file objects).
For instance, back then trying to install some packages would crash under pypy due to file exhaustion, because setuptools leaked fds all over the place: https://bitbucket.org/pypy/pypy/issues/878 https://bitbucket.org/pypy/pypy/issues/878
The stdlib also did: https://bugs.python.org/issue13133 https://bugs.python.org/issue13133
- catern 6y ago>the GC is triggered by memory pressure and that's it. Because it knows nothing about fd pressure, So why not teach it about fd pressure? There's no reason not to teach a GC about fd pressure - the reason that's rarely done is that language implementers generally don't care that much about the system interface - they just link against libc and call it done.
- dragonwriter 6y ago> You can't even define a custom hashing function All objects subject to GC have a memory impact, very few of them have an fd impact. You either have to separately track those that impact fds, or do a lot (up to 100%) unnecessary work in response to fd pressure. Even if user code can disable the fd pressure response, you've got to carry the baggage of a GC with extra code to handle it around. It's very easy to see “teaching a GC about fd pressure” as not worth the time and effort for the main runtime GC for a general purpose language. If you had a facility for swapping in special purpose GCs for particular applications, building a specialty GC that added fd awareness might be a net win for some apps, but...
- catern 6y ago>It's very easy to see “teaching a GC about fd pressure” as not worth the time and effort for the main runtime GC for a general purpose language. Yes, that's what I said: >language implementers generally don't care that much about the system interface - they just link against libc and call it done. There's no research done in this area (at least I don't know about any). I'm sure if people cared about it, they would find good solutions. But they mostly don't care about the system interface - which leads to the problems that this article is talking about. (Not that Python is on the fore-front of CS research - it's downstream of more innovative languages - but if those languages cared and had done research about the system interface, they would have come up with good ideas that might flow down to Python)