7 ms·

It's an obvious-enough "don't" that it's one of the stated reasons for universal laziness in Haskell. Simon Peyton Jones has said that he's committed to keeping

It's an obvious-enough "don't" that it's one of the stated reasons for universal laziness in Haskell. Simon Peyton Jones has said that he's committed to keeping everything in GHC lazy to keep them honest about side effects. I think the idea is basically that if something has side effects, it will become painful to use immediately due to Haskell's laziness.

willtim 9y ago
In Haskell it is still of course pure functional programming to have a lazy stream of IO actions. The types make it clear when and where the IO is happening, which makes reasoning about effectful streaming much easier.
- kerkeslager 9y ago
  I don't think it really makes sense to call the way side effects happen in Haskell "lazy". The way I think about it is that it's not so much that your program executes a lazy stream of side effects, it's that your program doesn't execute side effects at all. Instead, your program returns a pure structure which describes how side effects should be executed, and the runtime executes a program based on this description. By the time the runtime is executing that program, the pure structure which describes how the side effects should be executed has already been forced, so the side effects happen eagerly within that context. That said, I'm not much of a Haskell-er, so I'd be interested to hear from more experienced Haskell programmers on whether there's a better way to think about this.
  - willtim 9y ago
    > I don't think it really makes sense to call the way side effects happen in Haskell "lazy". I didn't actually say this. I was suggesting a lazy chain of IO actions for such problems. For example, ListT a IO, instead of a regular lazy list with hidden side-effects. The Haskell types help make the distinction clear. > Instead, your program returns a pure structure which describes how side effects should be executed Yes exactly, the pure structure can still be computed lazily on demand, providing IO actions. My favourite Hackage streaming library for Haskell is "streaming" and it can work exactly this way.
    - Tarean 9y ago
      Worth noting that ListTs bind isn't associative for most base monads including IO and it is deprecated for a reason. Also, by it's nature ListT can't stream IO unless you use lazy IO aka lazyRead aka unsafeInterleaveIO which deserves its name.
      - willtim 9y ago
        I am assuming "ListT done right" not the deprecated one (which is not even a chain of actions). Here is a definition, to be completely clear: newtype ListT m a = ListT (m (Maybe (a, ListT m a))) It is a nice simple example of an effectful streaming type. > Also, by it's nature ListT can't stream IO This is only true of the deprecated one, which is also misnamed. The community is rightly re-using the name for the behaviour shown above. See list-t on hackage, or the implementation that comes with Pipes.
  - wz1000 9y ago
    Haskell does have lazy side effects(unsafeInterleaveIO and its derivatives), which is a major source of complaints about the haskell standard library(lazy IO). There is a huge library ecosystem(conduit, pipes, streaming, machines, what have you) to solve this problem. For example, this code will fail because the file handle is closed prematurely by withFile as hGetContents returns immediately without actually reading the file, deferring that part until the data is forced. main = do contents <- withFile "foo.txt" ReadMode hGetContents putStrLn $ length contents
    - taeric 9y ago
      I'm assuming there is a more idiomatic (nay, correct?) way of doing this in Haskell?
      - danidiaz 9y ago
        One possible solution is to take a "beautiful fold" http://hackage.haskell.org/package/foldl-1.3.7/docs/Control-Foldl-ByteString.html#v:length http://hackage.haskell.org/package/foldl-1.3.7/docs/Control-... and connect it to an effectful stream of bytes http://hackage.haskell.org/package/pipes-bytestring-2.1.6/docs/Pipes-ByteString.html#v:fromHandle http://hackage.haskell.org/package/pipes-bytestring-2.1.6/do... inside the callback of "withFile". Not unlike the Streams/Collectors framework in Java.
        taeric 9y ago
        But... does that work? Doesn't that leave you open to having something evaluate the end of the "withFile" before you evaluated something in the callback? That is, the only reason what you said is accurate, is because you said "effectful stream of bytes" meaning "eagerly effectful." Right? If you do all of that lazily, you can just as easily shoot yourself in the foot here. (Please nobody take this as a damning criticism of Haskell. I'm not intending it that way.)
        danidiaz 9y ago
        The streams/sinks work in the IO monad and that induces an ordering of the effects, yes. Within the callback, you can still do things like filtering or taking only a part of the stream. Returning the stream itself from the "withFile" and trying to run it later still is still a bug. I like this approach better that conventional Haskell lazy I/O, which I personally find quite confusing.
jes5199 9y ago
I’ve had a Haskell program fail in exactly the same way, though - removing a debug statement caused a program to start crashing - even though the lazy functions were completely pure. We had a numeric counter that was enumerating the frame number of a simulation. When we removed the debug statement, we didn’t realize that no other code ever evaluated the counter - so instead of resolving each X+1 into an integer, Haskell kept it as a ever-growing nested series of unevaluated thunks: 1+1+1+1+1+1+1+1+1+1+... , and our efficient little program started filling up all its available RAM and crashing.
- taeric 9y ago
  Just, wow. I'm curious if there are mechanisms that could guard against growing strings of unevaluated thunks. Would be an interesting GC problem. One where you free up space by forcing a maximum depth of unevaluated space. (You'd obviously have to treat any "infinitely extra" computations versus the "do this on top of that" ones. But that shouldn't be undoable, should it?)
  - jpittis 9y ago
    Relevant blog post and HN comment thread about tracking space leaks in Haskell. https://news.ycombinator.com/item?id=10263964 https://news.ycombinator.com/item?id=10263964
  - zenhack 9y ago
    Yeah, this is Haskell's biggest footgun by far. The language gives you tools to control evaluation order (including some neat libraries that let you parallelize things without much disruption to the logic of your program), but there are no silver bullets. A good rule of thumb is to by default mark fields for basic types like Int, Bool, etc. as strictly evaluated, and leave larger structures (trees, lists, etc) lazy. But you still need to be careful. The compiler trying to silently fix things is probably a bad idea. The current behavior is at least easy to understand; I'd hate to have a program that's working because the compiler could figure out that it could make something strict, and then I bump something mostly unrelatrd such that the optimizer can't be sure anymore, so I get a space leak. Also, unintended evaluation can potentially cause high memory use as well (e.g. [1..1000000]), so the compiler also has to be careful about introducing excessive memory use. The compiler does do some strictness analysis, but it's a hard problem. I like the way idris does things -- strict by default, laziness controlled by the type system, and some nice support for automatic coercions.
hota_mazi 9y ago
How does laziness keep you honest about side effects exactly? If anything, eager evaluation will reveal the bug faster.
- kerkeslager 9y ago
  Side effects aren't a bug, in most programs, but they are impure (and purity is the main goal of Haskell). So if Simon Peyton Jones or someone else introduces code that causes side effects, in an eagerly-evaluated pure language like ML, it might not be obvious, because programs that use the impure code can behave as expected. But if you introduce side effects to a lazy language like Haskell, the side effects will happen at the wrong time, or not at all, making it immediately obvious that your code has side effects, and SPJ can replace the impure code with pure code that returns a description of the side effects, which are then evaluated by the runtime, as Haskell intended side effects to be handled.

willtim 9y ago

In Haskell it is still of course pure functional programming to have a lazy stream of IO actions. The types make it clear when and where the IO is happening, which makes reasoning about effectful streaming much easier.

kerkeslager 9y ago
I don't think it really makes sense to call the way side effects happen in Haskell "lazy". The way I think about it is that it's not so much that your program executes a lazy stream of side effects, it's that your program doesn't execute side effects at all. Instead, your program returns a pure structure which describes how side effects should be executed, and the runtime executes a program based on this description. By the time the runtime is executing that program, the pure structure which describes how the side effects should be executed has already been forced, so the side effects happen eagerly within that context. That said, I'm not much of a Haskell-er, so I'd be interested to hear from more experienced Haskell programmers on whether there's a better way to think about this.
- willtim 9y ago
  > I don't think it really makes sense to call the way side effects happen in Haskell "lazy". I didn't actually say this. I was suggesting a lazy chain of IO actions for such problems. For example, ListT a IO, instead of a regular lazy list with hidden side-effects. The Haskell types help make the distinction clear. > Instead, your program returns a pure structure which describes how side effects should be executed Yes exactly, the pure structure can still be computed lazily on demand, providing IO actions. My favourite Hackage streaming library for Haskell is "streaming" and it can work exactly this way.
  - Tarean 9y ago
    Worth noting that ListTs bind isn't associative for most base monads including IO and it is deprecated for a reason. Also, by it's nature ListT can't stream IO unless you use lazy IO aka lazyRead aka unsafeInterleaveIO which deserves its name.
    - willtim 9y ago
      I am assuming "ListT done right" not the deprecated one (which is not even a chain of actions). Here is a definition, to be completely clear: newtype ListT m a = ListT (m (Maybe (a, ListT m a))) It is a nice simple example of an effectful streaming type. > Also, by it's nature ListT can't stream IO This is only true of the deprecated one, which is also misnamed. The community is rightly re-using the name for the behaviour shown above. See list-t on hackage, or the implementation that comes with Pipes.
- wz1000 9y ago
  Haskell does have lazy side effects(unsafeInterleaveIO and its derivatives), which is a major source of complaints about the haskell standard library(lazy IO). There is a huge library ecosystem(conduit, pipes, streaming, machines, what have you) to solve this problem. For example, this code will fail because the file handle is closed prematurely by withFile as hGetContents returns immediately without actually reading the file, deferring that part until the data is forced. main = do contents <- withFile "foo.txt" ReadMode hGetContents putStrLn $ length contents
  - taeric 9y ago
    I'm assuming there is a more idiomatic (nay, correct?) way of doing this in Haskell?
    - danidiaz 9y ago
      One possible solution is to take a "beautiful fold" http://hackage.haskell.org/package/foldl-1.3.7/docs/Control-Foldl-ByteString.html#v:length http://hackage.haskell.org/package/foldl-1.3.7/docs/Control-... and connect it to an effectful stream of bytes http://hackage.haskell.org/package/pipes-bytestring-2.1.6/docs/Pipes-ByteString.html#v:fromHandle http://hackage.haskell.org/package/pipes-bytestring-2.1.6/do... inside the callback of "withFile". Not unlike the Streams/Collectors framework in Java.
      - taeric 9y ago
        But... does that work? Doesn't that leave you open to having something evaluate the end of the "withFile" before you evaluated something in the callback? That is, the only reason what you said is accurate, is because you said "effectful stream of bytes" meaning "eagerly effectful." Right? If you do all of that lazily, you can just as easily shoot yourself in the foot here. (Please nobody take this as a damning criticism of Haskell. I'm not intending it that way.)
        danidiaz 9y ago
        The streams/sinks work in the IO monad and that induces an ordering of the effects, yes. Within the callback, you can still do things like filtering or taking only a part of the stream. Returning the stream itself from the "withFile" and trying to run it later still is still a bug. I like this approach better that conventional Haskell lazy I/O, which I personally find quite confusing.

jes5199 9y ago

I’ve had a Haskell program fail in exactly the same way, though - removing a debug statement caused a program to start crashing - even though the lazy functions were completely pure. We had a numeric counter that was enumerating the frame number of a simulation. When we removed the debug statement, we didn’t realize that no other code ever evaluated the counter - so instead of resolving each X+1 into an integer, Haskell kept it as a ever-growing nested series of unevaluated thunks: 1+1+1+1+1+1+1+1+1+1+... , and our efficient little program started filling up all its available RAM and crashing.

taeric 9y ago
Just, wow. I'm curious if there are mechanisms that could guard against growing strings of unevaluated thunks. Would be an interesting GC problem. One where you free up space by forcing a maximum depth of unevaluated space. (You'd obviously have to treat any "infinitely extra" computations versus the "do this on top of that" ones. But that shouldn't be undoable, should it?)
- jpittis 9y ago
  Relevant blog post and HN comment thread about tracking space leaks in Haskell. https://news.ycombinator.com/item?id=10263964 https://news.ycombinator.com/item?id=10263964
- zenhack 9y ago
  Yeah, this is Haskell's biggest footgun by far. The language gives you tools to control evaluation order (including some neat libraries that let you parallelize things without much disruption to the logic of your program), but there are no silver bullets. A good rule of thumb is to by default mark fields for basic types like Int, Bool, etc. as strictly evaluated, and leave larger structures (trees, lists, etc) lazy. But you still need to be careful. The compiler trying to silently fix things is probably a bad idea. The current behavior is at least easy to understand; I'd hate to have a program that's working because the compiler could figure out that it could make something strict, and then I bump something mostly unrelatrd such that the optimizer can't be sure anymore, so I get a space leak. Also, unintended evaluation can potentially cause high memory use as well (e.g. [1..1000000]), so the compiler also has to be careful about introducing excessive memory use. The compiler does do some strictness analysis, but it's a hard problem. I like the way idris does things -- strict by default, laziness controlled by the type system, and some nice support for automatic coercions.

hota_mazi 9y ago

How does laziness keep you honest about side effects exactly? If anything, eager evaluation will reveal the bug faster.

kerkeslager 9y ago
Side effects aren't a bug, in most programs, but they are impure (and purity is the main goal of Haskell). So if Simon Peyton Jones or someone else introduces code that causes side effects, in an eagerly-evaluated pure language like ML, it might not be obvious, because programs that use the impure code can behave as expected. But if you introduce side effects to a lazy language like Haskell, the side effects will happen at the wrong time, or not at all, making it immediately obvious that your code has side effects, and SPJ can replace the impure code with pure code that returns a description of the side effects, which are then evaluated by the runtime, as Haskell intended side effects to be handled.