3 ms·

I know that this is true in general, but this case here seems tricky to me. Look at the following line: > yield (s[len(s)//2:], s[:len(s)//2]) I'm aware that

I know that this is true in general, but this case here seems tricky to me. Look at the following line: > yield (s[len(s)//2:], s[:len(s)//2]) I'm aware that the generator is being called lazily, but here we're actually calling each yield twice - once the left side is thrown away, once the right side downstream. Now, are you still sure that left and right is generated only once?

icebraining 11y ago
Oh, right, I missed that. No, it's actually generating both every time. CPython can't eliminate one of the sides, since the calls (either to len() or to the slicing method of 's') might have side effects. In this case, I think it'd make more sense to divide gen_splitted_his into two functions (or one which accepted a parameter), for generating one or the other.
- m_mueller 11y ago
  I have been toying around now with this second python example for a bit. Here's my conclusion: Not only is it not efficient, it doesn't even work. The generator cannot be rewound after it has been iterated over for lowercase - the uppercase version will never be executed. You essentially run into the problem described here: http://stackoverflow.com/questions/1271320/reseting-generator-object-in-python http://stackoverflow.com/questions/1271320/reseting-generato... . It only works once you actually generate twice > lc_his = gen_lc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10)))) > uc_his = gen_uc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10)))) It seems to me for this toy example, generators are simply not the right tool.
  - icebraining 11y ago
    Yes, but I think the author realizes that, which is why (s)he wrote above "this will produce an error, since running a generator function twice is not allowed, and this would not re-use the same items in memory anyway, which is kind of the main point with stream processing". The solution being proposed at the end is actually not using generators at all.
    - m_mueller 11y ago
      Oh I see, I missed that this "limitation" also applies for his DPI example that I was toying with.

icebraining 11y ago

Oh, right, I missed that. No, it's actually generating both every time. CPython can't eliminate one of the sides, since the calls (either to len() or to the slicing method of 's') might have side effects. In this case, I think it'd make more sense to divide gen_splitted_his into two functions (or one which accepted a parameter), for generating one or the other.

m_mueller 11y ago
I have been toying around now with this second python example for a bit. Here's my conclusion: Not only is it not efficient, it doesn't even work. The generator cannot be rewound after it has been iterated over for lowercase - the uppercase version will never be executed. You essentially run into the problem described here: http://stackoverflow.com/questions/1271320/reseting-generator-object-in-python http://stackoverflow.com/questions/1271320/reseting-generato... . It only works once you actually generate twice > lc_his = gen_lc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10)))) > uc_his = gen_uc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10)))) It seems to me for this toy example, generators are simply not the right tool.
- icebraining 11y ago
  Yes, but I think the author realizes that, which is why (s)he wrote above "this will produce an error, since running a generator function twice is not allowed, and this would not re-use the same items in memory anyway, which is kind of the main point with stream processing". The solution being proposed at the end is actually not using generators at all.
  - m_mueller 11y ago
    Oh I see, I missed that this "limitation" also applies for his DPI example that I was toying with.