3 ms·
My understanding is that category theory can be considered as abstract function theory. If we can prove that a function (e.g. Haskell compiler) is equivalent to
by apex3stoker 8y ago
My understanding is that category theory can be considered as abstract function theory. If we can prove that a function (e.g. Haskell compiler) is equivalent to automatic differentiation with some transformation, according to category theory, we can write Haskell code to train neural network, instead of writing extra graph building code in, say, Tensorflow. This paper basically shows this. http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf.
- formalsystem 8y agoYou may be interested in this paper too https://arxiv.org/abs/1711.10455 https://arxiv.org/abs/1711.10455 I think given a language with AD you already don't need to have a separate language for graph building and my understanding is that's one of the goals for the Julia team and I've seen other nice looking packages for AD in other languages * https://fluxml.ai/2019/02/07/what-is-differentiable-programming.html https://fluxml.ai/2019/02/07/what-is-differentiable-programm... * http://diffsharp.github.io/DiffSharp/examples-inversekinematics.html http://diffsharp.github.io/DiffSharp/examples-inversekinemat... I'm also very excited about being able to design neural nets using ideas from functional programming. I love the Keras API but I don't think it's going to be the obvious API a couple of years down the line.
- eru 7y agoAnd for a completely different application of differentiation to programming, see http://strictlypositive.org/diff.pdf http://strictlypositive.org/diff.pdf