6 ms·
Dynamic programming in Haskell: automatic memoization
- kccqzy 3y agoWhile this is a cool demo, I don't really like it even as a Haskell enthusiast. Anything that relies on compiler's eta reduction/expansion rules for performance is too subtle for my taste. I'd personally argue that more explicit code could be clearer.
- lgas 3y agoHow does it rely on the compiler's eta reduction/expansion rules? It doesn't seem to from my (decidedly intermediate) point of view.
- bojo 3y agoEta reduction/expansion has nothing to do with performance. It's a form of point free programming that allows you to abstract out intermediate variables and use partial application instead. f x = reverse (sort x) Becomes: f = reverse . sort The compiler reaches the same conclusion either way. Perhaps you are thinking of RULES pragmas? Which honestly you wouldn't usually reach for unless you really need to force the compiler to optimize in a particular way that is non-trivial.
- kccqzy 3y agoI might have used incorrect terminology but what I meant in my previous comment is that it should be obvious when the right hand side is evaluated. If a function has type declaration `f :: A -> B -> C` you would expect that its body only gets evaluated when two arguments are actually supplied. So evaluating the function itself using `seq` doesn't do anything. Nor does partial application of a single argument. To be absolutely clear, the expressions `seq f (putStrLn "hello")` and `seq (f undefined) (putStrLn "hello")` should both not evaluate anything and cause the string hello to be printed rather than raising an exception. However you can make it otherwise. That breaks the user's intuition about evaluation. Most of the time it's not a problem at all; I just personally don't like it and prefer to be more explicit. This often occurs in cases where the above hypothetical function `f` requires an intermediate value that's expensive to compute but only relies on the first of its two arguments. Then do you evaluate that expensive intermediate with a partial application of a single argument, or do you not? Neither is satisfactory and so this problem is best avoided. In your trivial example `reverse . sort` the compiler will inline the dot operator and make it `\x -> reverse (sort x)` and I have no issue there.
- quchen 3y ago> If a function has type declaration `f :: A -> B -> C` you would expect that its body only gets evaluated when two arguments are actually supplied. That’s not correct, regardless of η expansion. The core of the issue is that this function really takes one argument, regardless of how in practice it is used as if it had two. > the expressions `seq f (putStrLn "hello")` and `seq (f undefined) (putStrLn "hello")` should both not evaluate anything and cause the string hello to be printed rather than raising an exception. That’s not how `seq` works, as per language report. The definition is very simple, and does not take into account any kinds of types (such as treating functions differently). The report demands that `seq ⊥ x = ⊥` [1], so if the first argument is ⊥, you get a crash. Whether `x` is also evaluated in this context is up to the compiler actually, but what certainly cannot happen is that a "hello" is printed, because that would require a non-⊥ value to be passed to the surrounding IO context, but the whole `seq` is ⊥, so that’s not possible. In GHC, seq is essentially a tag that helps the strictness analyzer, and hence the optimizer. There is one subtlety in GHC that makes it not respect η expansion, and that is strictness properties, namely that `seq ⊥ () = ⊥`, but `seq (\x -> ⊥ x) () = ()`, were you referring to that? (In my opinion this is a violation of the Haskell Report.) > This often occurs in cases where the above hypothetical function `f` requires an intermediate value that's expensive to compute but only relies on the first of its two arguments. Lookup tables work that way, yes! I’m using this technique for walking a certain distance on a Bezier curve for example (Code at [2], result picture at [3]) Call that academic and I’ll show you the DIN A1 sized CNC machine I control using that logic ;-P [1]: https://www.haskell.org/onlinereport/haskell2010/haskellch6.html#x13-1260006.2 https://www.haskell.org/onlinereport/haskell2010/haskellch6.... [2]: https://github.com/quchen/generative-art/blob/master/src/Geometry/Bezier.hs#L236 https://github.com/quchen/generative-art/blob/master/src/Geo... [3]: https://quchen.github.io/generative-art/generative-art-0.1.0.0/Geometry-Bezier.html#v:bezierSubdivideEquidistant https://quchen.github.io/generative-art/generative-art-0.1.0...
- kccqzy 3y agoThank you for a long comment. I should've been clearer in my original comment. The crux of the matter is really that the arity of a function should be clear from its type, but those functions violate that assumption. You might think it's not a valid assumption but in practice I think it's a matter of good style to enforce it. My point about seq is really about arity. If a function has arity 2 then partial application of ⊥ cannot cause any part of the function to be executed and therefore the function cannot inspect its argument and find that it is ⊥.
- consilient 3y ago> Eta reduction/expansion has nothing to do with performance. It affects inlining. GHC only inlines fully applied functions, where "fully applied" means applied to as many arguments as it was declared with. So in something like `g = fmap f` your first definition of `f` would never get inlined into g whereas your second definition might.
- throwaway81523 3y agoMaybe you mean using lazy evaluation as a memoization hack? That is a well known trick in Haskell though I agree it is in some ways cringy, relying on the evaluation strategy like that. But there are some Hackage packages that make it fairly convenient to use. I did a naive implementation myself and benchmarked it against an explicit STRef as a cache cell. I don't remember the result but it wasn't a huge (OOM+) difference iirc. Rather than using an array as the memo cache (which means you must know its size in advance), it's nicer to use an infinite branching tree.
- kccqzy 3y agoRight. Personally I'd be much happier using an STRef. The code is much more explicit.
- throwaway81523 3y agoThe trouble then is that you have to wrap the memoized function and everything that uses it in the ST monad. Lazy evaluation is completely pure.
- lincpa 3y ago[dead]
- moi2388 3y agoThis is great. I was just thinking about this last week.
- lewtds 3y agoCertain logic programming languages like SWI-Prolog [0] and Picat [1] have this built-in. [0]: https://www.swi-prolog.org/pldoc/man?section=tabling-memoize https://www.swi-prolog.org/pldoc/man?section=tabling-memoize [1]: http://retina.inf.ufsc.br/picat_guide/#x1-210001.5 http://retina.inf.ufsc.br/picat_guide/#x1-210001.5
- thesz 3y agoThat's main Haskell strength: there often is a library in Haskell for what is language feature in other languages.
- cubefox 3y agoNot sure this is a strength, it sounds like Java.
- EdwardDiego 3y agoCould well be a strength of Java ;)
- thesz 3y agoYes, it is a strength. You may want a new control operator. In case of, say, Java, you go and ask language developers. In case of Haskell, you write a function or a bunch of functions. "if-then-else" can be implemented in Haskell as a function, for one example. Myself, I created a lot of these control operators when I worked with Haskell. I also used a lot of operators defined by others.
- cubefox 3y agoThis is cool, but in practice there are restrictions on syntax and grammar which limit what you can do compared to native "syntactic sugar".
- deleted 3y ago[deleted]
- deleted 3y ago
- moomin 3y agoNo disrespect to the dude that I learned Haskell from, but a) he doesn’t achieve his original goal and b) the goal is much less practically useful than he implies. In practice an awful lot of the functions you want to memoize a) involve continuations and b) want some way of evicting things from the cache. Haskell has no end of cute tricks for doing deterministic functions, it’s just that deterministic functions are programming’s easy mode.
- hinkley 3y agoMemoization is also entry level DP, and too easily confused with caching (if I had a dollar for every time someone equated the two, I could buy a baseball bat to knock heads). If you want the power of dynamic programming you need to get at least as far as tabulation.
- hinkley 3y agoCowards. Respond if you have problems. I’ve just gone through removing “memoization” from two parts of our code that were difficult to understand even without the caching layer. The reason I did the second one is because the first one went so well I wanted to see if it was a fluke. In doing the first, what I did was replace depth first execution with a tree scan. At the end I had a set of actions that needed to happen, and why. Essentially a plan for the execution. That’s a great spot for a breakpoint if the code or a test are failing, but it also yields you a set with all of the duplicates removed, instead of a graph. And in this case the number of data operations to combine all of the data is n+1 instead of > nlog(n) based on the number of duplicates. It also pointed out other optimizations that were possible. In the second case it also surfaces a potential ordering problem, and the added wrinkle was that some warning messages were being repeated, quite confusing to other developers. Those could be reduced from eight to three. In both cases memory usage was reduced (and in the one case the code was a bottleneck for starting short lived processes) because you don’t cache anything that is used once. There is no clever trick you need to work out. It just works, and the mechanism is perfectly readable and reason-able.
- nequo 3y ago
- simiones 3y agoDoes this technique keep the memoization table in memory while the top level function is in scope (so essentially forever?) Or does it become unreachable once the initial call to fib from somewhere else finish?
- deleted 3y ago[deleted]
- byorgey 3y agoIt keeps the table in memory while the function is in scope.
- tsimionescu 3y agoI assume this could be fixed by using a lambda instead of a top level function if you only want to occasionally call such a function, and don't need the whole memoization table around all the time?
- consilient 3y agoYes, although you need to make sure it doesn't get optimized away.
- dermesser 3y agoThis appears very similar (my Haskell is a bit rusty) to the problem description in Real World OCaml (https://dev.realworldocaml.org/imperative-programming.html https://dev.realworldocaml.org/imperative-programming.html), although made easier by the use of non-pure functions there.
- EdwardDiego 3y ago> Magically, fib is now fast Any code you describe with "Magically" is going to bite you in the arse in about two months. Maybe three. Leave magic to the guy who also does balloon animals at kids parties.
- rowanG077 3y agoI can't disagree more with this. Running any modern language is magic. If you don't want to use magic. You only get to write assembly or maybe C. Not that I agree with this solution.
- EdwardDiego 3y agoYou could probably disagree more with it, if you really tried :D But it's a bit early in the evening for reductio ad absurdum. After all, we can take it further into absurdity, why aren't we all just programming in hex via a keypad in the front of the computer? Abstraction != magic. The one bit of SICP that really stuck in my brain was: "Programs must be written for people to read, and only incidentally for machines to execute" Magic makes the first part far harder. If I need to understand your deep magic to understand your code, well, that's making my life harder, and don't you want people to understand what you're doing and why? I apply this critique to every kind of magic in code. Annotations in Java frameworks (I've had so much fun with Spring proxy invocations over RPC via Camel), monkey patching and metaprogramming in dynamic languages (or as Zed Shaw referred to it in Ruby, chainsaw infanticide) ...looking at you Django... Oh, and of course weak type coercion hacks and Perl's (< 6) DWMI. I mean, once you decode the sigils, it's a good feeling because you "get it", but if I want to feel clever for "getting it", I'll do cryptic crosswords. I'd far rather workaday code be boring and obvious when I have to make a mental model of it in order to fix it. I like it when I can read your code after you've moved onto another company, and understand what the code is doing and why without having to inveigle you into an awkward Zoom call or having to rewrite the magic step by painful step after the magician left, just so it's accessible to others. I've used "memoizing intermediate steps" often in the past, but it doesn't need to be magic.
- 3y ago
- thesuperbigfrog 3y ago"Wouldn’t it be cool if we could just write the recursive function, and then have some generic machinery make it fast for us by automatically generating a memo table?" "In other words, we’d like a magic memoization function . . . Then we could just define our slow, recursive function normally, wave our magic memo wand over it, and get a fast version for free!" Perl has this exact functionality with Memoize: https://perldoc.perl.org/Memoize https://perldoc.perl.org/Memoize More programming languages should have this functionality built in.
- mrfox321 3y agosame with python: @functools.cache def fn(x): https://docs.python.org/3/library/functools.html#functools.cache https://docs.python.org/3/library/functools.html#functools.c...
- deleted 3y ago[deleted]
- dndn1 3y agoI develop calculang, a language for calculations. It converts concise formulae into Javascript. Since calculation formulae are usually inherently pure, a `--memo` flag to the compiler memoizes the lot. Many of my models are written in this recursive style and only work because of memoization. Sometimes other tricks are needed e.g. careful ordering of the calls, so that the memo is available before the stack breaks. My first calculang example is a bouncing ball: the position at t depends on speed and the position at previous t. With loops we'll usually discover the same thing, but it can be so much harder to follow. First-class memoization to proliferate this style is a really neat thing. [0] https://github.com/calculang/calculang https://github.com/calculang/calculang [1] bouncing ball code: https://github.com/declann/calculang-miscellaneous-models/blob/main/models/bounce/bounce.cul.js https://github.com/declann/calculang-miscellaneous-models/bl... [2] bouncing ball initial post: https://observablehq.com/@declann/calculang-bouncing-ball?collection=@declann/calculang https://observablehq.com/@declann/calculang-bouncing-ball?co... [3] if you can tolerate my slow-load WIP devtools and are on a Desktop, this is better, with buttons to navigate through the model development (only some of which are working now!): https://models-on-a-plane.pages.dev/stories/bounce/ https://models-on-a-plane.pages.dev/stories/bounce/