8 ms·

I’ve always felt any mention of reducers is a big distraction. They’re not really key to the idea of transducers which are almost entirely about pure sequence t

by thom 3y ago

I’ve always felt any mention of reducers is a big distraction. They’re not really key to the idea of transducers which are almost entirely about pure sequence to sequence transforms.

thom 3y ago
For the benefit of downvoters, let me explain myself: the 'reducing function' (also called a 'reducing step function' in other parts of the docs) isn't always what you'd think of as the normal arity-2 function passed to reduce itself. Take a look at the reducing functions in clojure.core - the 3-arity, innermost function returned inside all the transducers. They almost never do anything with result which would be the accumulator in a normal reduction - they just pass it on or return it. In fact, it wouldn't make much sense for transducers to do any sort of reduction because if they did, you'd no longer be doing a step-wise transform over a sequence, and you could just use normal function composition because you'd have a single value to pass forward. The reduced mechanism is only really used in transducers to short circuit, so even there the name clouds the meaning. The only time that something we'd recognise as reduction is happening is when you call transduce and specify a reducing function as its f argument. You'll have seen lots of examples with + as the reducing function because it handily returns 0 when called with no args, acts as identity with 1 arg, and reduces (actually reduces!) with two args. But instead look at sequence which can take an xform. It's implemented in Java code, but the inner reducing function it uses completely ignores the accumulator value! Everywhere you're using result inside a reducing function? It's just null and ignored. But sequence elsewhere stashes away the (transformed) output of the transducer stack and lets you have it back as a sequence. No reducing is happening, just transforming sequences. Elsewhere, into supports an xform argument - what's its reducing function? It's just conjing stuff back together! So to me, the interesting bits of transducers are that they give you an efficient and composable way of creating sequence-to-sequence transformations. Any and all mentions of reducers are necessary plumbing but slightly distracting from the core ideas.
- sesm 3y ago
  I'm not sure why the comment above received downvotes (I don't have enough karma to downvote). I agree that perhaps the term "reducing function" isn't entirely accurate. However, it's worth noting that transducers aren't strictly about sequence-to-sequence transformations. From its inception, the Clojure standard library was built upon the `seq` protocol, and all its collection functions were constructed using this protocol. If it were possible to implement the `seq` protocol on channels, it would address the issue. However, it's not feasible. When querying a channel about its next element, the only honest answer is, "I can't determine that without potentially waiting indefinitely until the next message comes in or the channel closes." Turning the `seq` protocol into an asynchronous IO-bound process would be problematic. This is why transducers are essential.
  - thom 3y ago
    Yes, I'm being pernickety about naming and then misusing it myself! I use 'sequence' in a very loose sense to mean any sequential stuff you might push through a stack of transducers one at a time. core.async does actually use all the reducer plumbing to fill up buffers and pass them around. But I still think the key intuition for transducers should be push-based, stepwise operation (with the option to accumulate state or short circuit, as you go). It definitely helps that you can perform the final step of whatever job you're doing and have full control over how your stuff is materialised, without any intermediate collections, but it still feels slightly secondary to me.

by thom 3y ago

I’ve always felt any mention of reducers is a big distraction. They’re not really key to the idea of transducers which are almost entirely about pure sequence to sequence transforms.

thom 3y ago

For the benefit of downvoters, let me explain myself: the 'reducing function' (also called a 'reducing step function' in other parts of the docs) isn't always what you'd think of as the normal arity-2 function passed to reduce itself. Take a look at the reducing functions in clojure.core - the 3-arity, innermost function returned inside all the transducers. They almost never do anything with result which would be the accumulator in a normal reduction - they just pass it on or return it. In fact, it wouldn't make much sense for transducers to do any sort of reduction because if they did, you'd no longer be doing a step-wise transform over a sequence, and you could just use normal function composition because you'd have a single value to pass forward. The reduced mechanism is only really used in transducers to short circuit, so even there the name clouds the meaning. The only time that something we'd recognise as reduction is happening is when you call transduce and specify a reducing function as its f argument. You'll have seen lots of examples with + as the reducing function because it handily returns 0 when called with no args, acts as identity with 1 arg, and reduces (actually reduces!) with two args. But instead look at sequence which can take an xform. It's implemented in Java code, but the inner reducing function it uses completely ignores the accumulator value! Everywhere you're using result inside a reducing function? It's just null and ignored. But sequence elsewhere stashes away the (transformed) output of the transducer stack and lets you have it back as a sequence. No reducing is happening, just transforming sequences. Elsewhere, into supports an xform argument - what's its reducing function? It's just conjing stuff back together! So to me, the interesting bits of transducers are that they give you an efficient and composable way of creating sequence-to-sequence transformations. Any and all mentions of reducers are necessary plumbing but slightly distracting from the core ideas.

sesm 3y ago
I'm not sure why the comment above received downvotes (I don't have enough karma to downvote). I agree that perhaps the term "reducing function" isn't entirely accurate. However, it's worth noting that transducers aren't strictly about sequence-to-sequence transformations. From its inception, the Clojure standard library was built upon the `seq` protocol, and all its collection functions were constructed using this protocol. If it were possible to implement the `seq` protocol on channels, it would address the issue. However, it's not feasible. When querying a channel about its next element, the only honest answer is, "I can't determine that without potentially waiting indefinitely until the next message comes in or the channel closes." Turning the `seq` protocol into an asynchronous IO-bound process would be problematic. This is why transducers are essential.
- thom 3y ago
  Yes, I'm being pernickety about naming and then misusing it myself! I use 'sequence' in a very loose sense to mean any sequential stuff you might push through a stack of transducers one at a time. core.async does actually use all the reducer plumbing to fill up buffers and pass them around. But I still think the key intuition for transducers should be push-based, stepwise operation (with the option to accumulate state or short circuit, as you go). It definitely helps that you can perform the final step of whatever job you're doing and have full control over how your stuff is materialised, without any intermediate collections, but it still feels slightly secondary to me.