6 ms·

I concur. I don't really get the name "transducer". It's just map+reduce? So... why do we need a new name and new libraries?

• tel 11y ago
  It's not really map+reduce. Transducers arise out of realizing that transformations between (possibly infinite) lists can be encoded as a "reducer transformations", "transducers". When you work in this encoding you automatically get lazy transformation (since everything goes through a function call) and the avoidance of intermediate data structures (since everything goes through a function call!). Moreover, there's a neat property of this encoding, a CPS encoding, which is that function composition still "works", it just works "backwards".
  • fulafel 11y ago
    Python iterators also give you lazy transformation and no intermediate data structures, no?
    • tel 11y ago
      Yeah, but they compose differently and are stateful. Transducers compose much like functions and are stateless.
      • danidiaz 11y ago
        Are transducers really stateless? Decoders and transducers that perform grouping operations need to keep some state.
        • tel 11y ago
          They produce and use state when they run (at least in the Fold transformer format), but that's a very different kind of statefulness than generators.
          • sametmax 11y ago
            "Stateful" doesn't mean "bad". I know functional programming is all the rage right now, but being stateless doesn't turn magically something into a better version of something stateful. If the stateful version is better integrated, have a heavier stateless version is not going to make your code better.
    • rob_smallshire 11y ago
      Yes, although transducers are more general than iterators. Iterators model lazy series from which you must 'pull' elements. Transducers can also be used with elements that are 'pushed', such as events coming from channels, or whatever. Essentially they decouple the transformation from the mechanisms by which input elements arrive and result elements are received, whilst still allowing separate transformations to be composed into more complex composite transformations.
      • sametmax 11y ago
        That's what I was saying : this is from somebody not knowing Python. Generators have a "send()" method to push data in it. We have yield from to delegate to sub coroutines. And generators can be classes for more advanced behaviors. All this lib does is to make something that already exists in the language, in a heavier and slower way.
        • danidiaz 11y ago
          > Generators have a "send()" method to push data in it. What would be the pythonic way of "mapping over" the inputs that are sent into a generator, or performing grouping operations on those inputs? For example, how to transform a generator that expects to be sent sets of strings into a generator that simply accepts a flat stream of strings?
          • rob_smallshire 11y ago
            You could use the python-transducer library <https://github.com/sixty-north/python-transducers> https://github.com/sixty-north/python-transducers> I mentioned in my other comments to do this. It supports using the same transducers with iterables and push-event coroutine generators.
          • sametmax 11y ago
            You don't, you chain small generators. Compositions of generators transforming data is the key.
    • sametmax 11y ago
      Exactly. This is a lib created by someone that didn't get, at all, the point of iteration in Python. We have lazy coroutine exactly for that. It's embeded in the language. You got itertools, sum, any, all et comprehensions to use them. Iteration is everything in Python. It's the core philosophy. We already got tools to do this in better, lighter way.