6 ms·

I have never understood the benefits of making syntactic sugar for lists in languages having higher order functions. Python kind-of gets away with it, but I sti

by pyrale 7y ago

I have never understood the benefits of making syntactic sugar for lists in languages having higher order functions. Python kind-of gets away with it, but I still don't get the benefits compared to using map and filter.

tsimionescu 7y ago
I think a huge benefit is the readability of more complex expressions (e.g. when you start introducing zip, groupBy, folds etc into the mix).
- 11235813213455 7y ago
  If you want to show some code examples of list comprehensions, I'd be glad to show the equivalent without list-comprehensions in JS (which had a rejected proposal for list-comphensions https://www.reddit.com/r/javascript/comments/5eottt/why_the_hell_did_they_have_to_remove_list/ https://www.reddit.com/r/javascript/comments/5eottt/why_the_..., no regret though), so we can compare the readability
  - omaranto 7y ago
    How about finding triples a<b<c<30 of pairwise relatively prime integers such that the sum of their cubes is a perfect cube? Here's a list comprehension in Python (notice the weird syntax Python uses for "let x = y", namely "for x in [y]" :P): [(a,b,c) for a in range(1,30) for b in range(a+1,30) if gcd(a,b)==1 for c in range(b+1,30) if gcd(a,c)==1 and gcd(b,c)==1 for d in [int((a**3+b**3+c**3)**(1.0/3)+0.5)] if a**3+b**3+c**3==d**3] I doubt that would look any nicer with higher order functions. Specially if you want to get the same efficiency: notice how here we test that gcd(a,b)==1 as soon as possible. In general efficient comprehensions involve pushing any filtering as early as possible, so with higher order functions you get an alternation of filter with map/flatmap. Here's a (probably lame) attempt at a Python higher order version: flatmap = lambda f, l: [x for t in l for x in f(t)] flatmap(lambda a: flatmap(lambda b: flatmap(lambda c: (lambda d: [(a,b,c)] if a**3+b**3+c**3==d**3 else [])(int((a**3+b**3+c**3)**(1.0/3)+0.5)) if gcd(a,c)==1 and gcd(b,c)==1 else [], range(b+1, 30)) if gcd(a,b)==1 else [], range(a+1,30)), range(1,30)) Maybe it would look nicer with some filters thrown in instead of abusing flatmap of empty lists, but it would be longer since you definitely need a flatmap for a and b, and either a flatmap or map for c. I'd love to see a cleaner version with higher order function.
    - tel 7y ago
      The same sensibilities apply in Haskell. These sort of longer, more structured problems are exactly when do notation comes out do a <- [1 .. 29] b <- [a+1 .. 29] guard $ gcd a b == 1 c <- [b+1 .. 29] guard $ gcd a c == 1 guard $ gcd b c == 1 let z = a ** 3 + b ** 3 + c ** 3 let d = z ** (1/3) + 0.5 guard $ z == d ** 3 return (a, b, c) Translation of this into HOFs is straightforward (the method guard uses helps tremendously), but it's not worth it most of the time. In Python where HOFs are so restricted it would be even worse.
      - omaranto 7y ago
        I agree. Also as minor nitpick, I doubt your Haskell typechecks (haven't tried it): you forgot to take the floor of d.
        tel 7y ago
        Probably, I didn't check it. I think the actual answer is messier: fromIntegral on a, b, and c.
    - omaranto 7y ago
      Here's the flatmap version with better formatting: flatmap(lambda a: flatmap(lambda b: flatmap(lambda c: (lambda d: [(a,b,c)] if a**3+b**3+c**3==d**3 else [])( int((a**3+b**3+c**3)**(1.0/3)+0.5)) if gcd(a,c)==1 and gcd(b,c)==1 else [], range(b+1, 30)) if gcd(a,b)==1 else [], range(a+1,30)), range(1,30))
    - 11235813213455 7y ago
      the a:b notation for JS is still in proposal phase https://github.com/tc39/proposal-slice-notation/pull/32 https://github.com/tc39/proposal-slice-notation/pull/32, but I'll use it below to create ranges I don't understand your `+0.5`, I've written below the transcription for your description [...1:30].map(a => [...a+1:30].filter(b => gcd(a,b)==1).map(b => [...b+1:30].filter(c => gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3)) .map(c => [a, b, c]) ) ).flat(2) Or another equivalent way: [...1:30].flatMap(a => [...a+1:30].filter(b => gcd(a,b)==1).flatMap(b => [...b+1:30].filter(c => gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) .flatMap(c => [[a, b, c]]) Or another way (generating first all the triples): [...1:30].flatMap(a => [...a+1:30].flatMap(b => [...b+1:30].flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) with https://github.com/tc39/proposal-iterator-helpers https://github.com/tc39/proposal-iterator-helpers, which would allow lazy evaluation, and make more sense than above (1:30).flatMap(a => (a+1:30).flatMap(b => (b+1:30).flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) without the slice-notation, it's really ugly (hence the proposal in the first link) Array.from({length: 30},(_,a)=>a+1) .flatMap(a=>Array.from({length: 30-a-1},(_,b)=>a+b+1) .flatMap(b => Array.from({length: 30-b-1},(_,c)=>b+c+1).flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3)))
  - tsimionescu 7y ago
    I'll show an example in C#, which has some of the best list comprehension syntax in my opinion (LINQ): from category in categories where category.Name.Contains("food") join product in products on product.CategoryID equals category.ID select new { Name = category.Name, MostExpensive = (from categoryProduct in productGroup orderby categoryProduct.Price descending select categoryProduct) .First() } In C#, with HOFs, this becomes: categories .Where( category => category.Name.Contains("food")) .GroupJoin(products, category => category.ID, product => product.CategoryID, (category, productGroup) => new { Name = category.Name, MostExpensive = productGroup.OrderByDescending( product => product.Price) .First() } ) The second higher-oder function form has 2 disadvantages that I see: (1) for every subclause, it must mention the lambda parameter again (with more filtering/ordering/mapping this becomes much more noticeable); (2) it is easier to mess up the order of parameters, especially for something like Join/GroupJoin, which takes 5 parameters.
    - 11235813213455 7y ago
      It's probably this in JS: categories .filter( category => category.Name.includes('food')) .flatMap(category => { const productGroup = products.filter(product => product.CategoryID === category.ID); return productGroup.length === 0 ? [] : [{ Name: category.Name, MostExpensive: products.reduce((a, b) => a.Price < b.Price ? b : a) }]; })
- dorfsmay 7y ago
  Once you have lazy iterators (eg: generator expressions in Python) you can split your complex iterator operations into multiple lines without losing on performance nor conciseness, and increase readability over even list compréhensions.
a-saleh 7y ago
What you get is multiple bound variables. Using only map and filter it would be harder to do i.e.: [(x*y) for x in range(0,5) for y in range(0,x)]
- tom_mellior 7y ago
  To get multiple variables you "just" have to nest lambdas. In Haskell, for instance: Prelude> concatMap (\x -> concatMap (\y -> [x * y]) [0..x]) [0..5] [0,0,1,0,2,4,0,3,6,9,0,4,8,12,16,0,5,10,15,20,25] Prelude> [x * y | x <- [0..5], y <- [0..x]] [0,0,1,0,2,4,0,3,6,9,0,4,8,12,16,0,5,10,15,20,25] I'd take the latter over the former any day. And yes, I did get an obscure type error on my first attempt at the concatMap version because I forgot to bracket x * y.
  - omaranto 7y ago
    I'd replace the inner concatMap (\y -> [x+y]) with a simple map (\y -> x+y), or, if I'm honest, with map (x+). Here's hoping HN doesn't mess up the asterisks... EDIT: I've replaced all asterisks with plus signs, because I couldn't figure out how to protect them from HN.
    - tom_mellior 7y ago
      Yeah, I realized that later. It's still more obscure than the version with the list comprehension. In particular, the place where x is introduced is at the very beginning, but the values it can be bound to are at the end of the expression, and that is much less clear than "x <- [0..5]". The rule for asterisks is that they must be followed by whitespace to show up as asterisks, so you just need to format your code nicely :-) x * y works.
namelosw 7y ago
Because for more advanced cases requires flatMap instead of just map and filter. You can only nested as a callback hell instead of streamlined syntactic sugar. Take an example in 'Learn You a Haskell': [ (a,b,c) | c <- [1..10], b <- [1..c], a <- [1..b], a^2 + b^2 == c^2] b depends on c while a depends on b. You do need a Monadic operation instead of Functor/Applicative to do it because of the dependency in the contexts. And then you can think of solving the eight queen problem[0]. It's far more readable if you have a syntactic sugar, I just found a Scala example: [1]. [0] https://en.wikipedia.org/wiki/Eight_queens_puzzle https://en.wikipedia.org/wiki/Eight_queens_puzzle [1] https://stackoverflow.com/questions/42789854/n-queens-in-scala https://stackoverflow.com/questions/42789854/n-queens-in-sca...
bjoli 7y ago
I'd say we should have a proper looping facility that allows for building lists without the fuzz of having special syntax. Common lisp has one. The ITER macro in CL has is user-extensible, as are racket's for-loops. For non-lazy languages, you either need some sort of loop fusion or a construct like transducers to get similar functionality.
- lispm 7y ago
  Some LOOP implementations are user-extensible, too.
  - bjoli 7y ago
    You made me read sbcls loop implementation. 1900 lines of macro code :)
    - lispm 7y ago
      In SBCL one could use ADD-LOOP-PATH... see also here: https://github.com/sshirokov/CLSQL/blob/master/sql/loop-extension.lisp https://github.com/sshirokov/CLSQL/blob/master/sql/loop-exte...
- omaranto 7y ago
  Common Lisp's iter or loop macros are special syntax for, among other things, building lists. I think maybe you should have said instead that it's nice when a language let's you add such special syntax in libraries. :)
  - bjoli 7y ago
    That is not quite what I want to get across. I think python's list comprehensions are great, but I think they are a shit replacement for a proper looping facility that does everything python's list comprehensions do and more. Racket's for loops are also macros that expand to tail recursive procedures within a letrec, but they are "standard syntax" (in the sense that they are included in the base library) just as loop is.

by pyrale 7y ago

tsimionescu 7y ago

I think a huge benefit is the readability of more complex expressions (e.g. when you start introducing zip, groupBy, folds etc into the mix).

11235813213455 7y ago
If you want to show some code examples of list comprehensions, I'd be glad to show the equivalent without list-comprehensions in JS (which had a rejected proposal for list-comphensions https://www.reddit.com/r/javascript/comments/5eottt/why_the_hell_did_they_have_to_remove_list/ https://www.reddit.com/r/javascript/comments/5eottt/why_the_..., no regret though), so we can compare the readability
- omaranto 7y ago
  How about finding triples a<b<c<30 of pairwise relatively prime integers such that the sum of their cubes is a perfect cube? Here's a list comprehension in Python (notice the weird syntax Python uses for "let x = y", namely "for x in [y]" :P): [(a,b,c) for a in range(1,30) for b in range(a+1,30) if gcd(a,b)==1 for c in range(b+1,30) if gcd(a,c)==1 and gcd(b,c)==1 for d in [int((a**3+b**3+c**3)**(1.0/3)+0.5)] if a**3+b**3+c**3==d**3] I doubt that would look any nicer with higher order functions. Specially if you want to get the same efficiency: notice how here we test that gcd(a,b)==1 as soon as possible. In general efficient comprehensions involve pushing any filtering as early as possible, so with higher order functions you get an alternation of filter with map/flatmap. Here's a (probably lame) attempt at a Python higher order version: flatmap = lambda f, l: [x for t in l for x in f(t)] flatmap(lambda a: flatmap(lambda b: flatmap(lambda c: (lambda d: [(a,b,c)] if a**3+b**3+c**3==d**3 else [])(int((a**3+b**3+c**3)**(1.0/3)+0.5)) if gcd(a,c)==1 and gcd(b,c)==1 else [], range(b+1, 30)) if gcd(a,b)==1 else [], range(a+1,30)), range(1,30)) Maybe it would look nicer with some filters thrown in instead of abusing flatmap of empty lists, but it would be longer since you definitely need a flatmap for a and b, and either a flatmap or map for c. I'd love to see a cleaner version with higher order function.
  - tel 7y ago
    The same sensibilities apply in Haskell. These sort of longer, more structured problems are exactly when do notation comes out do a <- [1 .. 29] b <- [a+1 .. 29] guard $ gcd a b == 1 c <- [b+1 .. 29] guard $ gcd a c == 1 guard $ gcd b c == 1 let z = a ** 3 + b ** 3 + c ** 3 let d = z ** (1/3) + 0.5 guard $ z == d ** 3 return (a, b, c) Translation of this into HOFs is straightforward (the method guard uses helps tremendously), but it's not worth it most of the time. In Python where HOFs are so restricted it would be even worse.
    - omaranto 7y ago
      I agree. Also as minor nitpick, I doubt your Haskell typechecks (haven't tried it): you forgot to take the floor of d.
      - tel 7y ago
        Probably, I didn't check it. I think the actual answer is messier: fromIntegral on a, b, and c.
  - omaranto 7y ago
    Here's the flatmap version with better formatting: flatmap(lambda a: flatmap(lambda b: flatmap(lambda c: (lambda d: [(a,b,c)] if a**3+b**3+c**3==d**3 else [])( int((a**3+b**3+c**3)**(1.0/3)+0.5)) if gcd(a,c)==1 and gcd(b,c)==1 else [], range(b+1, 30)) if gcd(a,b)==1 else [], range(a+1,30)), range(1,30))
  - 11235813213455 7y ago
    the a:b notation for JS is still in proposal phase https://github.com/tc39/proposal-slice-notation/pull/32 https://github.com/tc39/proposal-slice-notation/pull/32, but I'll use it below to create ranges I don't understand your `+0.5`, I've written below the transcription for your description [...1:30].map(a => [...a+1:30].filter(b => gcd(a,b)==1).map(b => [...b+1:30].filter(c => gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3)) .map(c => [a, b, c]) ) ).flat(2) Or another equivalent way: [...1:30].flatMap(a => [...a+1:30].filter(b => gcd(a,b)==1).flatMap(b => [...b+1:30].filter(c => gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) .flatMap(c => [[a, b, c]]) Or another way (generating first all the triples): [...1:30].flatMap(a => [...a+1:30].flatMap(b => [...b+1:30].flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) with https://github.com/tc39/proposal-iterator-helpers https://github.com/tc39/proposal-iterator-helpers, which would allow lazy evaluation, and make more sense than above (1:30).flatMap(a => (a+1:30).flatMap(b => (b+1:30).flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3))) without the slice-notation, it's really ugly (hence the proposal in the first link) Array.from({length: 30},(_,a)=>a+1) .flatMap(a=>Array.from({length: 30-a-1},(_,b)=>a+b+1) .flatMap(b => Array.from({length: 30-b-1},(_,c)=>b+c+1).flatMap(c => [[a, b, c]]))) .filter(([a, b, c]) => gcd(a,b)==1 && gcd(a,c)==1 && gcd(b,c)==1 && Number.isInteger((a**3+b**3+c**3)**(1/3)))
- tsimionescu 7y ago
  I'll show an example in C#, which has some of the best list comprehension syntax in my opinion (LINQ): from category in categories where category.Name.Contains("food") join product in products on product.CategoryID equals category.ID select new { Name = category.Name, MostExpensive = (from categoryProduct in productGroup orderby categoryProduct.Price descending select categoryProduct) .First() } In C#, with HOFs, this becomes: categories .Where( category => category.Name.Contains("food")) .GroupJoin(products, category => category.ID, product => product.CategoryID, (category, productGroup) => new { Name = category.Name, MostExpensive = productGroup.OrderByDescending( product => product.Price) .First() } ) The second higher-oder function form has 2 disadvantages that I see: (1) for every subclause, it must mention the lambda parameter again (with more filtering/ordering/mapping this becomes much more noticeable); (2) it is easier to mess up the order of parameters, especially for something like Join/GroupJoin, which takes 5 parameters.
  - 11235813213455 7y ago
    It's probably this in JS: categories .filter( category => category.Name.includes('food')) .flatMap(category => { const productGroup = products.filter(product => product.CategoryID === category.ID); return productGroup.length === 0 ? [] : [{ Name: category.Name, MostExpensive: products.reduce((a, b) => a.Price < b.Price ? b : a) }]; })
dorfsmay 7y ago
Once you have lazy iterators (eg: generator expressions in Python) you can split your complex iterator operations into multiple lines without losing on performance nor conciseness, and increase readability over even list compréhensions.

a-saleh 7y ago

What you get is multiple bound variables. Using only map and filter it would be harder to do i.e.: [(x*y) for x in range(0,5) for y in range(0,x)]

tom_mellior 7y ago
To get multiple variables you "just" have to nest lambdas. In Haskell, for instance: Prelude> concatMap (\x -> concatMap (\y -> [x * y]) [0..x]) [0..5] [0,0,1,0,2,4,0,3,6,9,0,4,8,12,16,0,5,10,15,20,25] Prelude> [x * y | x <- [0..5], y <- [0..x]] [0,0,1,0,2,4,0,3,6,9,0,4,8,12,16,0,5,10,15,20,25] I'd take the latter over the former any day. And yes, I did get an obscure type error on my first attempt at the concatMap version because I forgot to bracket x * y.
- omaranto 7y ago
  I'd replace the inner concatMap (\y -> [x+y]) with a simple map (\y -> x+y), or, if I'm honest, with map (x+). Here's hoping HN doesn't mess up the asterisks... EDIT: I've replaced all asterisks with plus signs, because I couldn't figure out how to protect them from HN.
  - tom_mellior 7y ago
    Yeah, I realized that later. It's still more obscure than the version with the list comprehension. In particular, the place where x is introduced is at the very beginning, but the values it can be bound to are at the end of the expression, and that is much less clear than "x <- [0..5]". The rule for asterisks is that they must be followed by whitespace to show up as asterisks, so you just need to format your code nicely :-) x * y works.

namelosw 7y ago

Because for more advanced cases requires flatMap instead of just map and filter. You can only nested as a callback hell instead of streamlined syntactic sugar. Take an example in 'Learn You a Haskell': [ (a,b,c) | c <- [1..10], b <- [1..c], a <- [1..b], a^2 + b^2 == c^2] b depends on c while a depends on b. You do need a Monadic operation instead of Functor/Applicative to do it because of the dependency in the contexts. And then you can think of solving the eight queen problem[0]. It's far more readable if you have a syntactic sugar, I just found a Scala example: [1]. [0] https://en.wikipedia.org/wiki/Eight_queens_puzzle https://en.wikipedia.org/wiki/Eight_queens_puzzle [1] https://stackoverflow.com/questions/42789854/n-queens-in-scala https://stackoverflow.com/questions/42789854/n-queens-in-sca...

bjoli 7y ago

I'd say we should have a proper looping facility that allows for building lists without the fuzz of having special syntax. Common lisp has one. The ITER macro in CL has is user-extensible, as are racket's for-loops. For non-lazy languages, you either need some sort of loop fusion or a construct like transducers to get similar functionality.

lispm 7y ago
Some LOOP implementations are user-extensible, too.
- bjoli 7y ago
  You made me read sbcls loop implementation. 1900 lines of macro code :)
  - lispm 7y ago
    In SBCL one could use ADD-LOOP-PATH... see also here: https://github.com/sshirokov/CLSQL/blob/master/sql/loop-extension.lisp https://github.com/sshirokov/CLSQL/blob/master/sql/loop-exte...
omaranto 7y ago
Common Lisp's iter or loop macros are special syntax for, among other things, building lists. I think maybe you should have said instead that it's nice when a language let's you add such special syntax in libraries. :)
- bjoli 7y ago
  That is not quite what I want to get across. I think python's list comprehensions are great, but I think they are a shit replacement for a proper looping facility that does everything python's list comprehensions do and more. Racket's for loops are also macros that expand to tail recursive procedures within a letrec, but they are "standard syntax" (in the sense that they are included in the base library) just as loop is.