6 ms·

This looks exciting, but I'm confused about the decision to add extra arity to collection-manipulating functions. "filter" that returns a collection or a transd

by unlogic 12y ago

This looks exciting, but I'm confused about the decision to add extra arity to collection-manipulating functions. "filter" that returns a collection or a transducer depending only on arity seems a little counter-intuitive.

swannodette 12y ago
It's a pretty well considered tradeoff in my opinion - no existing code breaks while at the same time all the transformation functions now have the same semantic interpretation when used in different contexts. The alternative would be to replicate the notion of `map`, `filter`, etc. again and again as occurred with reducers and higher level core.async operations on channels.
- unlogic 12y ago
  My argument is why not making it look like: (comp (transducer map inc) (transducer filter even?)) Of course, it's more typing and doesn't look as nice as single-arity map, but it goes along quite well with (partial map inc). I can be wrong but to this date Clojure didn't have a function that produces completely different things based on arity, did it?
  - swannodette 12y ago
    Not sure how what you're suggesting could possibly work unless transducer has a table of fn to transducer - this doesn't sound like a good idea. I recommend looking at the implementations which have landed in Clojure master to see what I'm talking about.
    - malisper 12y ago
      You could just make them transducer-map, transducer-filter, etc. There is no reason the exact same name should be used for two different things. It is a lot harder to tell the difference between (map inc) and (map inc xs) then it is to tell the difference between (transducer-map inc) and (map inc xs).
- taliesinb 12y ago
  We just did exactly the same thing in the Wolfram Language, for similar reasons (we called these things "operator forms" rather than "transducers") [0] One major side effect has been to mitigate the kinds of heavy nesting you see in functional languages like WL and Clojure. Personally I think the resulting code resembles the phrase structure of English much more closely. It's a huge readability win. The original motivations for operator forms were in fact writing Queries [1] against Datasets [2], for which you want to represent complex operations independent of their execution. [0] http://reference.wolfram.com/language/guide/FunctionCompositionAndOperatorForms.html http://reference.wolfram.com/language/guide/FunctionComposit... [1] http://reference.wolfram.com/language/ref/Query.html http://reference.wolfram.com/language/ref/Query.html [2] http://reference.wolfram.com/language/ref/Dataset.html http://reference.wolfram.com/language/ref/Dataset.html
  - richhickey 12y ago
    I'm not seeing anything that looks like a reducing function transformer there. That all looks like variants of ordinary function composition, currying and partial application. Is there someplace that shows 'operator forms' acting as functions with this signature: (x->a->x)->(x->a->x)?
    - taliesinb 12y ago
      WL doesn't yet have a laziness/streaming/reducing framework, but the prototype we're working on uses 'operator forms' like Select, Map, GroupBy and so on in the way you describe. I don't think the exact details are the same, because our operators don't actually evaluate to transformers (they remain totally symbolic). Rather, the conversion of composed operators to an actual reducer pipeline happens lazily 'at the right time', which I think will make optimization a bit easier to express.
      - jamii 12y ago
        So if I understand correctly, you would arrive at a symbolic expression like (->> (map f) (map g) x) and directly rewrite it to (->> (map (f . g)) x), rather than having to manipulate the resulting data-structures?
- kenko 12y ago
  It seems very weird to me; it looks like currying (or partial application) but actually you're getting two fundamentally different things. Partially applying `map` to one argument with `partial` (or just doing `#(map f %)`) gets you one thing; non-partially applying map to one argument gets you something totally different.

by unlogic 12y ago

swannodette 12y ago

It's a pretty well considered tradeoff in my opinion - no existing code breaks while at the same time all the transformation functions now have the same semantic interpretation when used in different contexts. The alternative would be to replicate the notion of `map`, `filter`, etc. again and again as occurred with reducers and higher level core.async operations on channels.

unlogic 12y ago
My argument is why not making it look like: (comp (transducer map inc) (transducer filter even?)) Of course, it's more typing and doesn't look as nice as single-arity map, but it goes along quite well with (partial map inc). I can be wrong but to this date Clojure didn't have a function that produces completely different things based on arity, did it?
- swannodette 12y ago
  Not sure how what you're suggesting could possibly work unless transducer has a table of fn to transducer - this doesn't sound like a good idea. I recommend looking at the implementations which have landed in Clojure master to see what I'm talking about.
  - malisper 12y ago
    You could just make them transducer-map, transducer-filter, etc. There is no reason the exact same name should be used for two different things. It is a lot harder to tell the difference between (map inc) and (map inc xs) then it is to tell the difference between (transducer-map inc) and (map inc xs).
taliesinb 12y ago
We just did exactly the same thing in the Wolfram Language, for similar reasons (we called these things "operator forms" rather than "transducers") [0] One major side effect has been to mitigate the kinds of heavy nesting you see in functional languages like WL and Clojure. Personally I think the resulting code resembles the phrase structure of English much more closely. It's a huge readability win. The original motivations for operator forms were in fact writing Queries [1] against Datasets [2], for which you want to represent complex operations independent of their execution. [0] http://reference.wolfram.com/language/guide/FunctionCompositionAndOperatorForms.html http://reference.wolfram.com/language/guide/FunctionComposit... [1] http://reference.wolfram.com/language/ref/Query.html http://reference.wolfram.com/language/ref/Query.html [2] http://reference.wolfram.com/language/ref/Dataset.html http://reference.wolfram.com/language/ref/Dataset.html
- richhickey 12y ago
  I'm not seeing anything that looks like a reducing function transformer there. That all looks like variants of ordinary function composition, currying and partial application. Is there someplace that shows 'operator forms' acting as functions with this signature: (x->a->x)->(x->a->x)?
  - taliesinb 12y ago
    WL doesn't yet have a laziness/streaming/reducing framework, but the prototype we're working on uses 'operator forms' like Select, Map, GroupBy and so on in the way you describe. I don't think the exact details are the same, because our operators don't actually evaluate to transformers (they remain totally symbolic). Rather, the conversion of composed operators to an actual reducer pipeline happens lazily 'at the right time', which I think will make optimization a bit easier to express.
    - jamii 12y ago
      So if I understand correctly, you would arrive at a symbolic expression like (->> (map f) (map g) x) and directly rewrite it to (->> (map (f . g)) x), rather than having to manipulate the resulting data-structures?
kenko 12y ago
It seems very weird to me; it looks like currying (or partial application) but actually you're getting two fundamentally different things. Partially applying `map` to one argument with `partial` (or just doing `#(map f %)`) gets you one thing; non-partially applying map to one argument gets you something totally different.