3 ms·

So like partially applied functions in Haskell?

by rowbin 3y ago

lgrapenthin 3y ago
Not even by far
mrkeen 3y ago
Probably just plain old functions & laziness. If you want to interleave IO into it then probably a library like conduit.
- adityaathalye 3y ago
  Plain old functions & eager evaluation & a bit more awesome sauce. Given transducers, we can compose mutually independent parts at will: - Data source (sequence, stream, channel, socket etc.) - Data sink (sequence, stream, channel, socket etc.) - Data transformer (function of any value -> any other value) - Data transformation process (mapping, filtering, reducing etc.) - Some process control (we can transduce finite data (of course) as well as streams, and also have optional early termination in either case. I'm not sure about first-class support for other methods like backpressure.) e.g. read numbers off a Kafka topic, FizzBuzz them, and send them to another Kafka topic, OR slurp numbers from file on disk, FizzBuzz them, and push into an in-memory queue. But each time, you don't have to rewrite your core fizzbuzz function, nor your `(map fizzbuzz)` definition. cf. https://www.evalapply.org/posts/n-ways-to-fizzbuzz-in-clojure/index.html#transducery-buzz https://www.evalapply.org/posts/n-ways-to-fizzbuzz-in-clojur...
  - throwaway858 3y ago
    I'm not sure why the parent was downvoted, this sounds exactly like the Haskell conduit library (or indeed plain laziness if you don't need IO).
    - waffletower 3y ago
      I can't downvote, but might have as the first sentence is an over-simplication and misunderstanding -- particularly as laziness for collections has always been available in clojure.core. Clojure transducers offer an optimization orthogonal to collections best summed above with: "transducers allow you to define steps in collection processing _per item_ rather than having collection processing as a series of transformations of collections". Yes, transducers can be viewed as somewhat of an analog to the Haskell conduit library (as discussed here several years ago: https://hypirion.com/musings/haskell-transducers https://hypirion.com/musings/haskell-transducers). However, I think the detractors coming from strongly typed languages are decidedly missing much of the generalization of the transducer model, particularly those conflating transducers exclusively with streams.
      - throwaway858 3y ago
        Thanks for this link. It seems to confirm things: "aren’t Conduits and Transducers then equivalent (isomorphic)? I am pretty sure they are." I view this as a good sign. When two independent parties arrive at the same design it is usually an indication that they have discovered a universal and principled solution. I consider the "conduit" library to be one of Haskell's "killer features", and sorely miss having something like it when working in other languages. Maybe when Haskellers dismiss clojure transducers as being "just like conduit" it comes from a place of jealousy? I've seen several articles and discussions over the years of clojure transducers that take place outside of clojure communities and are aimed at the wider programming public, praising the benefits of it. But I've never seen conduit discussed outside of Haskell communities.
bmacho 3y ago
You probably are thinking of normal functions, and not partially applied ones (which are also just normal functions that we get a special way totally unrelated here). Also I don't think they can reproduce Blammo! Our gnome is now packaging together incoming items into bundles of three, caching them in the interim while the bundle is not complete yet. But if we close the input prematurely, it will acknowledge and produce the incomplete bundle: (>!! b 4) (>!! b 5) (close! b) ; Value: [4 5]
crdrost 3y ago
It's not, this is that Lisp thing where you can check the length of your argument list and do something completely different when you don't get enough arguments. In this case `(map f)` notices that it was told to map a function but not told what to map it over, and so it decides to give you a new function. If this were a partially applied function the signature would be `[x] -> [y]` where `f: x -> y` would pick out the specifics. But this is actually a totally different signature, isomorphic to `x -> [y]`, the signature of generators. Specifically `(map f)` generates what in Haskell would be `\x -> [f x]`. However the type is not quite that straightforward for historical and compositional reasons; it is actually ∀z. (y -> z -> z) -> x -> z -> z With the implementation being here \handle x -> handle (f x) This is a sort of enhanced map that can do filtering because it uses concatMap (also known as >>=, “bind in the list monad”) to combine. So to `(filter pred)` you would have the isomorphic versions, \x -> if pred x then [x] else [] \handle x rest -> if pred x then handle x rest else rest This also leads to an important nitpick for the article in question, a strictly better mental model of a transducer is not that it maps conveyor belts to conveyor belts, since that has more power than transducers do. (For instance, reverse is not a transducer.) But rather that it maps individual items on a conveyor belt, to their own conveyor belts on the first conveyor belt, then mashes them all together into one effective conveyor belt. So filter will either map an object to a singleton conveyor belt containing that thing, or an empty conveyor belt. You can implement `dupIf pred x = if pred x then [ x, x] else [x]` as a transducer too, `handle x (handle x rest)`. Conveyor belt that either has one or two elements on it. You can potentially put an infinite conveyor belt inside your conveyor belts and make a chunk of the input unreachable, although Clojure is strict so I have the feeling this will just run out of memory?
- rowbin 3y ago
  Thanks, that helped
slowmovintarget 3y ago
Transducers are functions that return transformed reducing functions.
lmm 3y ago
They're like iteratees, but with awkward edge cases (particularly around error handling). If you've used Conduit you'll have already had the positive experiences other comments are talking about - realising how powerful and general-purpose the abstraction is and using it for everything.

So like partially applied functions in Haskell?

by rowbin 3y ago

So like partially applied functions in Haskell?

lgrapenthin 3y ago

Not even by far

mrkeen 3y ago

Probably just plain old functions & laziness. If you want to interleave IO into it then probably a library like conduit.

adityaathalye 3y ago
Plain old functions & eager evaluation & a bit more awesome sauce. Given transducers, we can compose mutually independent parts at will: - Data source (sequence, stream, channel, socket etc.) - Data sink (sequence, stream, channel, socket etc.) - Data transformer (function of any value -> any other value) - Data transformation process (mapping, filtering, reducing etc.) - Some process control (we can transduce finite data (of course) as well as streams, and also have optional early termination in either case. I'm not sure about first-class support for other methods like backpressure.) e.g. read numbers off a Kafka topic, FizzBuzz them, and send them to another Kafka topic, OR slurp numbers from file on disk, FizzBuzz them, and push into an in-memory queue. But each time, you don't have to rewrite your core fizzbuzz function, nor your `(map fizzbuzz)` definition. cf. https://www.evalapply.org/posts/n-ways-to-fizzbuzz-in-clojure/index.html#transducery-buzz https://www.evalapply.org/posts/n-ways-to-fizzbuzz-in-clojur...
- throwaway858 3y ago
  I'm not sure why the parent was downvoted, this sounds exactly like the Haskell conduit library (or indeed plain laziness if you don't need IO).
  - waffletower 3y ago
    I can't downvote, but might have as the first sentence is an over-simplication and misunderstanding -- particularly as laziness for collections has always been available in clojure.core. Clojure transducers offer an optimization orthogonal to collections best summed above with: "transducers allow you to define steps in collection processing _per item_ rather than having collection processing as a series of transformations of collections". Yes, transducers can be viewed as somewhat of an analog to the Haskell conduit library (as discussed here several years ago: https://hypirion.com/musings/haskell-transducers https://hypirion.com/musings/haskell-transducers). However, I think the detractors coming from strongly typed languages are decidedly missing much of the generalization of the transducer model, particularly those conflating transducers exclusively with streams.
    - throwaway858 3y ago
      Thanks for this link. It seems to confirm things: "aren’t Conduits and Transducers then equivalent (isomorphic)? I am pretty sure they are." I view this as a good sign. When two independent parties arrive at the same design it is usually an indication that they have discovered a universal and principled solution. I consider the "conduit" library to be one of Haskell's "killer features", and sorely miss having something like it when working in other languages. Maybe when Haskellers dismiss clojure transducers as being "just like conduit" it comes from a place of jealousy? I've seen several articles and discussions over the years of clojure transducers that take place outside of clojure communities and are aimed at the wider programming public, praising the benefits of it. But I've never seen conduit discussed outside of Haskell communities.

bmacho 3y ago

You probably are thinking of normal functions, and not partially applied ones (which are also just normal functions that we get a special way totally unrelated here). Also I don't think they can reproduce Blammo! Our gnome is now packaging together incoming items into bundles of three, caching them in the interim while the bundle is not complete yet. But if we close the input prematurely, it will acknowledge and produce the incomplete bundle: (>!! b 4) (>!! b 5) (close! b) ; Value: [4 5]

crdrost 3y ago

It's not, this is that Lisp thing where you can check the length of your argument list and do something completely different when you don't get enough arguments. In this case `(map f)` notices that it was told to map a function but not told what to map it over, and so it decides to give you a new function. If this were a partially applied function the signature would be `[x] -> [y]` where `f: x -> y` would pick out the specifics. But this is actually a totally different signature, isomorphic to `x -> [y]`, the signature of generators. Specifically `(map f)` generates what in Haskell would be `\x -> [f x]`. However the type is not quite that straightforward for historical and compositional reasons; it is actually ∀z. (y -> z -> z) -> x -> z -> z With the implementation being here \handle x -> handle (f x) This is a sort of enhanced map that can do filtering because it uses concatMap (also known as >>=, “bind in the list monad”) to combine. So to `(filter pred)` you would have the isomorphic versions, \x -> if pred x then [x] else [] \handle x rest -> if pred x then handle x rest else rest This also leads to an important nitpick for the article in question, a strictly better mental model of a transducer is not that it maps conveyor belts to conveyor belts, since that has more power than transducers do. (For instance, reverse is not a transducer.) But rather that it maps individual items on a conveyor belt, to their own conveyor belts on the first conveyor belt, then mashes them all together into one effective conveyor belt. So filter will either map an object to a singleton conveyor belt containing that thing, or an empty conveyor belt. You can implement `dupIf pred x = if pred x then [ x, x] else [x]` as a transducer too, `handle x (handle x rest)`. Conveyor belt that either has one or two elements on it. You can potentially put an infinite conveyor belt inside your conveyor belts and make a chunk of the input unreachable, although Clojure is strict so I have the feeling this will just run out of memory?

rowbin 3y ago
Thanks, that helped

slowmovintarget 3y ago

Transducers are functions that return transformed reducing functions.

lmm 3y ago

They're like iteratees, but with awkward edge cases (particularly around error handling). If you've used Conduit you'll have already had the positive experiences other comments are talking about - realising how powerful and general-purpose the abstraction is and using it for everything.