4 ms·

Maybe I misunderstood, but if you're suggesting the generator function open/close the resource, then I disagree. The caller has the responsibility of safely ope

• cheez 6y ago
  I agree, this unnecessarily complicates library-type code.
• softwaredoug 6y ago
  Then the caller must know they are receiving a generator dependent on the files lifetime. Pretty brittle IMO. Receiving a list wouldn’t depend on the files lifetime. This causes client code to be pretty error prone as you can very easily think you’re saving off a list built from a file, but in reality you hold a generator. Client code closes the file, the generator is used much later, and blammo, crash! NOT doing a context manager forces clients to work harder at guessing lifetimes. So you can’t just replace lists with generators. This break the uniform access principal. Client code IMO shouldn’t need to think about whether they got a generator or an interable container.
  • xapata 6y ago
    > Then the caller must know they are receiving a generator dependent on the files lifetime How would they not? The caller provides the file as an argument to the generator function. Further, the caller decides when to close the file, presumably via leaving a context and after the generator is consumed.
    • Groxx 6y ago
      it's not too hard to write code like: with open(...) as f: lines = process(f) # do stuff with lines ^ if `process` returns a generator, the file will be closed without having done anything.
      • xapata 6y ago
        Put `do stuff with lines` inside the context. Where it belongs.
        • Groxx 6y ago
          and hold open a resource longer than necessary? or when looping over lines multiple times? it doesn't unambiguously belong in there. often, yes, but not always.
          • hansvm 6y ago
            As the caller you can always choose to write lines=list(process(f)) if your use case requires it. If process(f) returned a list though, you wouldn't be able to avoid the potentially large memory allocation if you only needed to iterate over the results.
            • Groxx 6y ago
              Sure. It's entirely possible. Do you `list(...)` literally everywhere you want a list, whether or not it's currently a list? Extreme-defensive-programming like that tends to be extremely rare in my experience. Not nonexistent, but I'd be willing to bet that if you took a random stack overflow or github line of code, it wouldn't do this.
              • xapata 6y ago
                No. I put that in places where I receive a generator and want a list. But, there are plenty of instances of what you're talking about. Take a look at the standard library. Itertools often creates an iterator as the first operation, not knowing whether it received an iterator or some other kind of iterable. https://github.com/python/cpython/blob/384621c42f9102e31ba2c47feba144af09c989e5/Modules/itertoolsmodule.c#L965 https://github.com/python/cpython/blob/384621c42f9102e31ba2c... If you search for `list` in the CPython code, you'll find many examples of what you're asking about. Similarly, Pandas often makes a copy of a Series or DataFrame. It might be inefficient, but it keeps the API (more) simple.