3 ms·
Recommending papers I have read: - Rebuilding Racket on Chez Scheme (experience report). A very applied and very readable report on an impressive project to re
by Athas 7y ago
Recommending papers I have read:
- Rebuilding Racket on Chez Scheme (experience report). A very applied and very readable report on an impressive project to re-target a significant existing compiler. Especially recommended if you normally don't like reading academic papers, since this reads more like a very good technical report.
- Efficient differentiable programming in a functional array-processing language. A cool technique for applying automatic differentiation to a small functional language. Surprising how well it seems to work in practice. Easy to read. Also, it's the first paper I have seen that actually contains an empirical comparison against Futhark (albeit only sequential code)!
- Selective applicative functors. A really cool technique for working with asynchronous effects. The kind of thing that is obvious (and obviously a good idea) in retrospect, but nobody put together the pieces before. Probably requires good Haskell knowledge to understand.
Papers I have not read yet but which look interesting:
- Lambda Calculus with Algebraic Simplification for Reduction Parallelization by Equational Reasoning. Writing parallel reductions with nontrivial operators has so far mostly been a case of thinking really hard, and then maybe proving associativity and commutativity by hand. I'm curious whether this paper can provide a more systematic approach.
- Demystifying differentiable programming: shift/reset the penultimate backpropagator. Automatic differentiation from a functional perspective is usually interesting.
- pwm 7y agoI would add "Higher-order type-level programming in Haskell" which will bring unsaturated type families, enabling currying at the type level.
- tempguy9999 7y agoWhat does "differentiable programming" mean? When I've looked up the term before it seems to relate to differentiation in the conventional mathematical, numerical sense. Is this something more?
- Athas 7y agoI haven't read the paper yet, but I think it's exactly about differentiating (semi-)arbitrary functions written in a conventional-ish programming language.
- keithalewis 7y agoSee https://github.com/keithalewis/epsilon https://github.com/keithalewis/epsilon for my approach to solving this problem.
- lonelappde 7y agoIt's a new name for computing neural network weights, by analogy to "linear programming" , where the program is optimizing a mathematical loss function with approximate real-number parameters (as opposed to a program that computes a precise result with Boolean /integer logic (such as a web server or a database)). Mathematically, it's analysis on parameters. Confusingly and beautifully, there is also "mechanical differentiation" on the algebra of data of types, another computer application of the same abstract mathematical concept of differentiation: https://en.m.wikibooks.org/wiki/Haskell/Zippers#Mechanical_Differentiation https://en.m.wikibooks.org/wiki/Haskell/Zippers#Mechanical_D... Read about both at http://conal.net/blog/tag/derivative http://conal.net/blog/tag/derivative
- tempguy9999 7y agoTa for the NN stuff. I have come across differentiation of ADTs and very strange it is too. I can't understand how it works, it's so weird. I'll try to follow your links. Thanks!