5 ms·

The article elaborates on this point a bit: > Like pipes in UNIX, the functions are clever enough to be performant when composed together–we don’t traverse the

The article elaborates on this point a bit: > Like pipes in UNIX, the functions are clever enough to be performant when composed together–we don’t traverse the whole list and generate a new list each time, each item is generated on demand. In fact, due to stream fusion, the code will be compiled into one fast loop. I imagine it is hard to do the same optimization in a non-lazy language.

klipt 6y ago
It's interesting that he says you also need purity to do the optimization, but Unix pipes are not necessarily pure.
- garethjrowlands 6y ago
  Unix pipes don't do stream fusion, which is the optimisation to which he was referring.
  - klipt 6y ago
    So if I put (x1, x2...) through pipes f and g, instead of calculating all the f(x)s first followed by all the g(f(x))s, it calculates g(f(x1)), then g(f(x2)), etc. How is that order of operations different to stream fusion? Why does stream fusion require more purity than Unix pipes?
    - newen 6y ago
      Let's say there's a global mutable variable M (doesn't have to global, can be x1.M = M, x2.M = M, for example, but easier to think with global) which is used by and modified when f(x) or g(y) runs. Then, the order of operations would matter and so you get different results with and without stream fusion.
      - klipt 6y ago
        Right, but couldn't the same thing happen with Unix pipes if the components write to a global file? You can do the same optimization Unix pipes does without language enforced purity, just with a caveat that functions sharing state might behave weirdly when fused.
        newen 6y ago
        You could... I guess language designers generally avoid those types of optimizations since it can bring about unexpected results. As in, if syntactically the code is using one order of operations and semantically the code is using another order of operations, even if it's documented, lots of programmers would stumble into difficult to find bugs and be surprised by this behavior. Or worse, just get wrong results and never even find out it's wrong.

klipt 6y ago

It's interesting that he says you also need purity to do the optimization, but Unix pipes are not necessarily pure.

garethjrowlands 6y ago
Unix pipes don't do stream fusion, which is the optimisation to which he was referring.
- klipt 6y ago
  So if I put (x1, x2...) through pipes f and g, instead of calculating all the f(x)s first followed by all the g(f(x))s, it calculates g(f(x1)), then g(f(x2)), etc. How is that order of operations different to stream fusion? Why does stream fusion require more purity than Unix pipes?
  - newen 6y ago
    Let's say there's a global mutable variable M (doesn't have to global, can be x1.M = M, x2.M = M, for example, but easier to think with global) which is used by and modified when f(x) or g(y) runs. Then, the order of operations would matter and so you get different results with and without stream fusion.
    - klipt 6y ago
      Right, but couldn't the same thing happen with Unix pipes if the components write to a global file? You can do the same optimization Unix pipes does without language enforced purity, just with a caveat that functions sharing state might behave weirdly when fused.
      - newen 6y ago
        You could... I guess language designers generally avoid those types of optimizations since it can bring about unexpected results. As in, if syntactically the code is using one order of operations and semantically the code is using another order of operations, even if it's documented, lots of programmers would stumble into difficult to find bugs and be surprised by this behavior. Or worse, just get wrong results and never even find out it's wrong.