4 ms·

The top link on /r/haskell right now [1] is someone complaining that their program runs out of space due to a subtle interaction between laziness and IO. I thin

The top link on /r/haskell right now [1] is someone complaining that their program runs out of space due to a subtle interaction between laziness and IO. I think it's safe to say that we have tried laziness as the default and have learned that it's the wrong default, because it plays havoc with space and with IO. [1] https://www.reddit.com/r/haskell/comments/5h6emf/haskell_run_out_of_memory/ https://www.reddit.com/r/haskell/comments/5h6emf/haskell_run...

tome 10y ago
No, that's not laziness per se, that's "lazy IO", something somewhat different. EDIT: It's not even a lazy IO problem since no lazy IO functions are used there.
- cousin_it 10y ago
  The problem is laziness. Lazy IO just makes it happen with files, which is more noticeable. In regular lazy code it happens with memory instead (space leaks).
  - fbreduc 10y ago
    these are not problems, only small details that you have to be careful of..not a large tradeoff to pay
    - marcosdumay 10y ago
      That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of. To tell you the true, I can't understand why the code the OP talks about has a problem, and I can't even reproduce the problem (and never saw any of the problems of lazy IO in practice either). But I also can not say the code has no problem, and that is a big issue with the language.
      - lomnakkus 10y ago
        > That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of. I think there's a significant difference between undefined behavior (hello security flaws!) and the program crashing in a (reasonably) well-defined way.
  - lomnakkus 10y ago
    EDIT: I was wrong. It seems to actually just be a hideously inefficient in-memory representation of the data (akin to List[Char]) perhaps plus a space leak. That's what I get for not reading the code carefully...
kqr 10y ago
Look, I fixed their code in so many ways: main = runConduitRes -- dealing with finite resources ( sourceFileBS "input.txt" -- read input.txt as binary data .| decodeC utf8 -- decode assuming UTF-8 .| linesC -- split into lines .| mapC parseList -- parse each line into list of text .| mapC (get 5) -- get sixth element of list .| catMaybeC -- discard lines with no sixth element .| encodeC utf8 -- encode as UTF-8 .| sinkFileBS "output.txt" -- dump into output.txt ) And it's still very functional, if not more so! This will run in constant space and linear time, it will buffer reasonably, it will not leak file handles, it will gracefully clean up on exceptions, it will not crash when it fails to parse something correctly, and it makes the encoding assumption explicit (you cannot split into lines unless you know the encoding). Dealing with I/O is not hard when you use the correct primitives.
- tdb7893 10y ago
  It seems to me that part of the problem he was having is that he didn't really understand how these methods were being implemented internally. I know that all languages inevitably have these problems but how do the amount of leaky abstractions compare in Haskell to other languages?
  - kqr 10y ago
    I don't think Haskell is worse than any other language in that regard. What perhaps sets it apart is how much of its functionality is implemented in third-party libraries, which may be difficult for a beginner to come to grips with. "Why should I download a library to use efficient arrays? Shouldn't they be built in, like in Python?" I can come up with two explanations for this reliance on third-party libraries. 1) Haskell has always been a quickly evolving language attracting research-minded people which in turn go on to develop really cool libraries that are much better than conventional ways of doing things. The interpretation of this explanation is that it's simply not possible to keep the standard library up to date with the latest library developments. It may also be the case that 2) Haskell has always been a really powerful language capable of offloading important tasks to libraries. What would need to be built-in functionality in other languages can be implemented as libraries with no sort of special treatment in Haskell, so people do it that way because they can, and because it keeps the base simple.
KirinDave 10y ago
I'm not sure why we keep pretending Haskell is supposed to work well. It's half prorgramming environment, half research environment. This shows in MANY ways. And don't get me wrong, the Haskell team tries, but ... There really isn't a serious attempt to build an industry-friendly programming language with functional features and laziness as a default. You'd make many different decisions than Haskell: - No more language extensions, you'd fix the feature set. - Better transparency on memory growth and GCs in tooling - It'd be easier not to generate a ton of garbage, so the GC could be designed such that very large working sets could be handled with low latency. - You wouldn't structure IO the way the have. Modern programs have demands for many, many side effects and threading an IO monad through every place they can touch is common practice, but not useful. A reduced power version of IO that would let developers push events to an IO-enabled functional reactor would be in the stdlib. - Monad Transformers would almost certainly not be included, preferring: http://lambda-the-ultimate.org/node/4786 http://lambda-the-ultimate.org/node/4786 We shouldn't hold Haskell as the perfect expression of functional and lazy programming. It was never meant to be and it shows. It's just the tool we have right now to go to bat with.
- chrisdone 10y ago
  I believe PureScript solves most of these problems.
  - cousin_it 10y ago
    Thanks! After your comment I finally got around to checking out PureScript, and it looks nice (aside from the crazy tooling requirements which are typical for 2016).
lmm 10y ago
Sure. I'll be moving to Idris as soon as it matures, since by enforcing totality you gain the advantages of laziness without the problems. In the meantime Haskell may well be the best option for various circumstances even with that flaw though. (And note that strict evaluation in non-total languages brings its own problems)

tome 10y ago

No, that's not laziness per se, that's "lazy IO", something somewhat different. EDIT: It's not even a lazy IO problem since no lazy IO functions are used there.

cousin_it 10y ago
The problem is laziness. Lazy IO just makes it happen with files, which is more noticeable. In regular lazy code it happens with memory instead (space leaks).
- fbreduc 10y ago
  these are not problems, only small details that you have to be careful of..not a large tradeoff to pay
  - marcosdumay 10y ago
    That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of. To tell you the true, I can't understand why the code the OP talks about has a problem, and I can't even reproduce the problem (and never saw any of the problems of lazy IO in practice either). But I also can not say the code has no problem, and that is a big issue with the language.
    - lomnakkus 10y ago
      > That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of. I think there's a significant difference between undefined behavior (hello security flaws!) and the program crashing in a (reasonably) well-defined way.
- lomnakkus 10y ago
  EDIT: I was wrong. It seems to actually just be a hideously inefficient in-memory representation of the data (akin to List[Char]) perhaps plus a space leak. That's what I get for not reading the code carefully...

kqr 10y ago

Look, I fixed their code in so many ways: main = runConduitRes -- dealing with finite resources ( sourceFileBS "input.txt" -- read input.txt as binary data .| decodeC utf8 -- decode assuming UTF-8 .| linesC -- split into lines .| mapC parseList -- parse each line into list of text .| mapC (get 5) -- get sixth element of list .| catMaybeC -- discard lines with no sixth element .| encodeC utf8 -- encode as UTF-8 .| sinkFileBS "output.txt" -- dump into output.txt ) And it's still very functional, if not more so! This will run in constant space and linear time, it will buffer reasonably, it will not leak file handles, it will gracefully clean up on exceptions, it will not crash when it fails to parse something correctly, and it makes the encoding assumption explicit (you cannot split into lines unless you know the encoding). Dealing with I/O is not hard when you use the correct primitives.

tdb7893 10y ago
It seems to me that part of the problem he was having is that he didn't really understand how these methods were being implemented internally. I know that all languages inevitably have these problems but how do the amount of leaky abstractions compare in Haskell to other languages?
- kqr 10y ago
  I don't think Haskell is worse than any other language in that regard. What perhaps sets it apart is how much of its functionality is implemented in third-party libraries, which may be difficult for a beginner to come to grips with. "Why should I download a library to use efficient arrays? Shouldn't they be built in, like in Python?" I can come up with two explanations for this reliance on third-party libraries. 1) Haskell has always been a quickly evolving language attracting research-minded people which in turn go on to develop really cool libraries that are much better than conventional ways of doing things. The interpretation of this explanation is that it's simply not possible to keep the standard library up to date with the latest library developments. It may also be the case that 2) Haskell has always been a really powerful language capable of offloading important tasks to libraries. What would need to be built-in functionality in other languages can be implemented as libraries with no sort of special treatment in Haskell, so people do it that way because they can, and because it keeps the base simple.

KirinDave 10y ago

I'm not sure why we keep pretending Haskell is supposed to work well. It's half prorgramming environment, half research environment. This shows in MANY ways. And don't get me wrong, the Haskell team tries, but ... There really isn't a serious attempt to build an industry-friendly programming language with functional features and laziness as a default. You'd make many different decisions than Haskell: - No more language extensions, you'd fix the feature set. - Better transparency on memory growth and GCs in tooling - It'd be easier not to generate a ton of garbage, so the GC could be designed such that very large working sets could be handled with low latency. - You wouldn't structure IO the way the have. Modern programs have demands for many, many side effects and threading an IO monad through every place they can touch is common practice, but not useful. A reduced power version of IO that would let developers push events to an IO-enabled functional reactor would be in the stdlib. - Monad Transformers would almost certainly not be included, preferring: http://lambda-the-ultimate.org/node/4786 http://lambda-the-ultimate.org/node/4786 We shouldn't hold Haskell as the perfect expression of functional and lazy programming. It was never meant to be and it shows. It's just the tool we have right now to go to bat with.

chrisdone 10y ago
I believe PureScript solves most of these problems.
- cousin_it 10y ago
  Thanks! After your comment I finally got around to checking out PureScript, and it looks nice (aside from the crazy tooling requirements which are typical for 2016).

lmm 10y ago

Sure. I'll be moving to Idris as soon as it matures, since by enforcing totality you gain the advantages of laziness without the problems. In the meantime Haskell may well be the best option for various circumstances even with that flaw though. (And note that strict evaluation in non-total languages brings its own problems)