3 ms·

> You could have a strict language with a growable stack This doesn't fix the more general problem of something that is generated by a (co-)recursive function

by wyager 6y ago

> You could have a strict language with a growable stack This doesn't fix the more general problem of something that is generated by a (co-)recursive function and consumed gradually. Non-strictness lets you "pause" the recursion until you need it. You can have a data structure that's bigger than memory (even infinite) and it's not a problem as long as you don't need all of it at once.

dwohnitmok 6y ago
Sure, but you can fix that with any number of streaming libraries in strict languages those times when you do actually need that. And importantly, in line with local reasoning, in a strict language, it will be very apparent when you need it. It won't take you by surprise. On the other hand, Haskell's default list and its associated functions are just enough to let you get by but have enough pitfalls that can you take you off-guard that in the end, for more complex operations, you just end up using a streaming library anyways (e.g. conduit, pipes, streamly, etc.).
- wyager 6y ago
  > you can fix that with any number of streaming libraries in strict languages This is moving the goalposts a bit. Certainly you can do streaming in a strict language if you're willing to use libraries, but we're talking about language primitives. > you just end up using a streaming library anyways You generally only do this if you're interleaving effects with your streaming. I use guarded corecursion all the time, e.g. when I write an expression like index = zip [1..] Any strict language that supports something like `[1..]` has a lot more crap to deal with, and usually has to implement something like overloading list expressions to create iterators, which have their own problems.
  - dwohnitmok 6y ago
    > This is moving the goalposts a bit. I don't think so. My original comment was focusing on stack overflow errors, but my wider thesis is this: By and large, strict languages trade off some level of syntactic convenience for easier, local reasoning about performance. And I am willing to accept that hit. So for example, yeah sure maybe you have to do a mechanical transformation of a recursive function to either use a trampoline or an explicit stack or rewrite with an iterative loop, but that's an acceptable trade-off for me. Stated another way: The cases where a lazy language can cause weird performance issues are harder to debug than the cases where a strict language can cause weird performance issues. Indeed the fact that a library is all that is needed to patch up a lack of language primitives makes me more willing to accept the trade off of syntactic convenience. index = zip [1..] vs index = Stream.zip (Stream.from 1) or even conversions to and from lists if you really don't want streams elsewhere such as index xs = Stream.toList . Stream.zip startIndex $ streamedXs where startIndex = Stream.from 1 streamedXs = Stream.fromList xs is a sufficiently small price for me to pay to gain back local performance reasoning (I mean in this case I'd just write index with a fold to begin with in a strict language). > interleaving effects with your streaming But the issue that effects are supposed to be values in and of themselves (i.e. if I really take effects as a value seriously, why must my machinery change when I have a list of IO actions vs a list of integers?). This is really a symptom of the deeper issue is that there are a lot of things that should be strict for other reasons that can interfere with purely lazy lists or other structures because they basically represent this sort of weird interleaving of strict and lazy needs (this is why e.g. foldl' fails without strict data fields).
slaymaker1907 6y ago
This is a solved problem, see the Racket programming language. I think this may apply to most Scheme implementations actually.
- wyager 6y ago
  Can you explain how racket solves this?
  - soegaard 6y ago
    It's answered here: https://stackoverflow.com/q/49912204/23567 https://stackoverflow.com/q/49912204/23567
    - wyager 6y ago
      The answers there seem to suggest racket just allows large amounts of space to be allocated to the stack. This doesn't "solve" the problem at all. It's just a tweak to kick the can down the road a bit. Non-strict languages fundamentally change the way one can think about function context; unlimited guarded corecursion via thunks is a canonical example.

by wyager 6y ago

dwohnitmok 6y ago

Sure, but you can fix that with any number of streaming libraries in strict languages those times when you do actually need that. And importantly, in line with local reasoning, in a strict language, it will be very apparent when you need it. It won't take you by surprise. On the other hand, Haskell's default list and its associated functions are just enough to let you get by but have enough pitfalls that can you take you off-guard that in the end, for more complex operations, you just end up using a streaming library anyways (e.g. conduit, pipes, streamly, etc.).

wyager 6y ago
> you can fix that with any number of streaming libraries in strict languages This is moving the goalposts a bit. Certainly you can do streaming in a strict language if you're willing to use libraries, but we're talking about language primitives. > you just end up using a streaming library anyways You generally only do this if you're interleaving effects with your streaming. I use guarded corecursion all the time, e.g. when I write an expression like index = zip [1..] Any strict language that supports something like `[1..]` has a lot more crap to deal with, and usually has to implement something like overloading list expressions to create iterators, which have their own problems.
- dwohnitmok 6y ago
  > This is moving the goalposts a bit. I don't think so. My original comment was focusing on stack overflow errors, but my wider thesis is this: By and large, strict languages trade off some level of syntactic convenience for easier, local reasoning about performance. And I am willing to accept that hit. So for example, yeah sure maybe you have to do a mechanical transformation of a recursive function to either use a trampoline or an explicit stack or rewrite with an iterative loop, but that's an acceptable trade-off for me. Stated another way: The cases where a lazy language can cause weird performance issues are harder to debug than the cases where a strict language can cause weird performance issues. Indeed the fact that a library is all that is needed to patch up a lack of language primitives makes me more willing to accept the trade off of syntactic convenience. index = zip [1..] vs index = Stream.zip (Stream.from 1) or even conversions to and from lists if you really don't want streams elsewhere such as index xs = Stream.toList . Stream.zip startIndex $ streamedXs where startIndex = Stream.from 1 streamedXs = Stream.fromList xs is a sufficiently small price for me to pay to gain back local performance reasoning (I mean in this case I'd just write index with a fold to begin with in a strict language). > interleaving effects with your streaming But the issue that effects are supposed to be values in and of themselves (i.e. if I really take effects as a value seriously, why must my machinery change when I have a list of IO actions vs a list of integers?). This is really a symptom of the deeper issue is that there are a lot of things that should be strict for other reasons that can interfere with purely lazy lists or other structures because they basically represent this sort of weird interleaving of strict and lazy needs (this is why e.g. foldl' fails without strict data fields).

slaymaker1907 6y ago

This is a solved problem, see the Racket programming language. I think this may apply to most Scheme implementations actually.

wyager 6y ago
Can you explain how racket solves this?
- soegaard 6y ago
  It's answered here: https://stackoverflow.com/q/49912204/23567 https://stackoverflow.com/q/49912204/23567
  - wyager 6y ago
    The answers there seem to suggest racket just allows large amounts of space to be allocated to the stack. This doesn't "solve" the problem at all. It's just a tweak to kick the can down the road a bit. Non-strict languages fundamentally change the way one can think about function context; unlimited guarded corecursion via thunks is a canonical example.