12 ms·

> Our team's big mistake was believing blog posts that swept laziness problems under the rug I linked a blog post[1]. I wouldn't say it sweeps anything under

by tome 2y ago

> Our team's big mistake was believing blog posts that swept laziness problems under the rug I linked a blog post[1]. I wouldn't say it sweeps anything under the rug. In fact it takes laziness out from under the rug and shows you exactly how to deal with it. If you think that approach wouldn't have solved your problem then perhaps you could elaborate why. [1] http://h2.jaguarpaw.co.uk/posts/make-invalid-laziness-unrepresentable/ http://h2.jaguarpaw.co.uk/posts/make-invalid-laziness-unrepr...

treesurgeon 2y ago
That's great work and a terrific post. We had lots of legacy subcomponents that permeated throughout the app's data structures, and as a practical matter, we couldn't reopen all that existing code. Some of that legacy code crossed organizational boundaries, so eliminating laziness throughout simply wasn't do-able. So given all the above, there are two reasonable conclusions, which aren't contradictory: 1) If we could have reworked lots of existing code, we could have eliminated space leaks via @tome's approach. 2) Haskell's laziness is fundamentally misaligned with projects like ours, which has cross-organization code, long-running iterative algorithms, and complex data structures. In desperation we used deepseq, which everyone knows is bad, and shortly thereafter we cut our losses by switching platforms.
- tome 2y ago
  Thanks! Although I'd be interested in debating the topic at length (and I also have a longstanding offer to resolve anyone's space leak problems) I don't want to be That Haskell Guy^{TM} who won't let the topic go, so I'll leave it up to you whether you want to continue the debate. I will say however, that this most recent comment is significantly less pessimistic than your original one (https://news.ycombinator.com/item?id=41170695 https://news.ycombinator.com/item?id=41170695) and I don't think I disagree with it (except if it subtly implies that Haskell must use laziness everywhere -- that's not true!).
  - treesurgeon 2y ago
    [dead]
    - tome 2y ago
      I'm reluctant to pick holes in what you describe because I know that I'm not getting a full picture and you have much more context than me, but with that disclaimer out of the way, I think I disagree fully with the details (despite, I think, agreeing with the broad picture). Firstly, I disagree that laziness vs strictness is an engineering trade off. There are not domains where a strict language is the best fit and others where a lazy language is the best fit. That's like saying there are domains where indentation sensitive languages are the best fit and others where languages that use braces are the best fit. Laziness, like indentation sensitivity, has some syntactic and semantic consequences and you just have to use it accordingly. Laziness, like indentation sensitivity, has costs and benefits, but there is no cost/benefit trade off! It's just a fairly arbitrary choice. It is a not a factor "when choosing a platform". There are no platforms where strictness is the best choice and no platforms where laziness is the best choice! Now, I appreciate that the Haskell community has not produced enough educational material and tooling for many users to feel fully comfortable with the correct style for avoiding space leaks. That is a somewhat different issue that the one you raised. However, once one does know those techniques, space leaks are basically a non-issue. I take the point that rewriting your application in Julia resolved the problem, but I find that hard to reconcile with the following: > We had lots of legacy subcomponents that permeated throughout the app's data structures, and as a practical matter, we couldn't reopen all that existing code If you could rewrite in Julia, surely you could also rewrite in Haskell and avoid all that legacy. The advice I'd give anyone struggling with space leaks today is to use the th-deepstrict[1] package to discover which of their long lived data types has been specified to allow space leaks, and then plug those leaks as explain in my article[2]. Easy. "But I shouldn't have to make an extra effort to avoid laziness!". I agree! I think strict data would be the correct default (but lazy function arguments). But also, in Python I shouldn't have to take extra care with scopes when creating lambdas, in Java I shouldn't have to deal with nullable object references, in X I shouldn't have to deal with Y. I agree! In the real world we just learn the ways of coping and get on with it. Haskell's laziness is a less severe impediment than either of the above issues, in my opinion. > SPJ's hair shirt presentation: "The next Haskell will be strict." Meh, this comes up a bunch because SPJ said it, but he only said it once and never again. It's not clear he really believes it. PureScript is basically "strict Haskell" and Haskellers aren't flocking to it because of that. You win something by moving to a strict language, and you also lose a lot. [1] https://hackage.haskell.org/package/th-deepstrict-0.1.0.0 https://hackage.haskell.org/package/th-deepstrict-0.1.0.0 [2] http://h2.jaguarpaw.co.uk/posts/nested-strict-data/ http://h2.jaguarpaw.co.uk/posts/nested-strict-data/

by tome 2y ago

treesurgeon 2y ago

That's great work and a terrific post. We had lots of legacy subcomponents that permeated throughout the app's data structures, and as a practical matter, we couldn't reopen all that existing code. Some of that legacy code crossed organizational boundaries, so eliminating laziness throughout simply wasn't do-able. So given all the above, there are two reasonable conclusions, which aren't contradictory: 1) If we could have reworked lots of existing code, we could have eliminated space leaks via @tome's approach. 2) Haskell's laziness is fundamentally misaligned with projects like ours, which has cross-organization code, long-running iterative algorithms, and complex data structures. In desperation we used deepseq, which everyone knows is bad, and shortly thereafter we cut our losses by switching platforms.

tome 2y ago
Thanks! Although I'd be interested in debating the topic at length (and I also have a longstanding offer to resolve anyone's space leak problems) I don't want to be That Haskell Guy^{TM} who won't let the topic go, so I'll leave it up to you whether you want to continue the debate. I will say however, that this most recent comment is significantly less pessimistic than your original one (https://news.ycombinator.com/item?id=41170695 https://news.ycombinator.com/item?id=41170695) and I don't think I disagree with it (except if it subtly implies that Haskell must use laziness everywhere -- that's not true!).
- treesurgeon 2y ago
  [dead]
  - tome 2y ago
    I'm reluctant to pick holes in what you describe because I know that I'm not getting a full picture and you have much more context than me, but with that disclaimer out of the way, I think I disagree fully with the details (despite, I think, agreeing with the broad picture). Firstly, I disagree that laziness vs strictness is an engineering trade off. There are not domains where a strict language is the best fit and others where a lazy language is the best fit. That's like saying there are domains where indentation sensitive languages are the best fit and others where languages that use braces are the best fit. Laziness, like indentation sensitivity, has some syntactic and semantic consequences and you just have to use it accordingly. Laziness, like indentation sensitivity, has costs and benefits, but there is no cost/benefit trade off! It's just a fairly arbitrary choice. It is a not a factor "when choosing a platform". There are no platforms where strictness is the best choice and no platforms where laziness is the best choice! Now, I appreciate that the Haskell community has not produced enough educational material and tooling for many users to feel fully comfortable with the correct style for avoiding space leaks. That is a somewhat different issue that the one you raised. However, once one does know those techniques, space leaks are basically a non-issue. I take the point that rewriting your application in Julia resolved the problem, but I find that hard to reconcile with the following: > We had lots of legacy subcomponents that permeated throughout the app's data structures, and as a practical matter, we couldn't reopen all that existing code If you could rewrite in Julia, surely you could also rewrite in Haskell and avoid all that legacy. The advice I'd give anyone struggling with space leaks today is to use the th-deepstrict[1] package to discover which of their long lived data types has been specified to allow space leaks, and then plug those leaks as explain in my article[2]. Easy. "But I shouldn't have to make an extra effort to avoid laziness!". I agree! I think strict data would be the correct default (but lazy function arguments). But also, in Python I shouldn't have to take extra care with scopes when creating lambdas, in Java I shouldn't have to deal with nullable object references, in X I shouldn't have to deal with Y. I agree! In the real world we just learn the ways of coping and get on with it. Haskell's laziness is a less severe impediment than either of the above issues, in my opinion. > SPJ's hair shirt presentation: "The next Haskell will be strict." Meh, this comes up a bunch because SPJ said it, but he only said it once and never again. It's not clear he really believes it. PureScript is basically "strict Haskell" and Haskellers aren't flocking to it because of that. You win something by moving to a strict language, and you also lose a lot. [1] https://hackage.haskell.org/package/th-deepstrict-0.1.0.0 https://hackage.haskell.org/package/th-deepstrict-0.1.0.0 [2] http://h2.jaguarpaw.co.uk/posts/nested-strict-data/ http://h2.jaguarpaw.co.uk/posts/nested-strict-data/