3 ms·
haskell, FP RNNs: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://hackage.haskell.org/
by noloblo 4y ago
haskell, FP
RNNs: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://hackage.haskell.org/package/grenade https://hackage.haskell.org/package/grenade
haskell, composing well FP, succintness is important for complex concepts involved in RNN, and using an RNN as another layer in Grenade is very elegant and this is enabled by the haskell's composability, type-safety and type-classes. This makes RNNs almost fun relative to how painful tensorflow and their imperative kin are to use in python. Albeit tensorflow is more advanced with more advanced features not yet available in weka, grenade, hlearn.
not all neural networks are equal imho: https://www.youtube.com/watch?v=sPjA6lS0GlQ https://www.youtube.com/watch?v=sPjA6lS0GlQ
>> what’s really nice about these functions is that they directly mirror the functions in Layers, in fact, this match works so well that we can embed a Network as a Layer in another Network. Grenade is a dependently typed, high level, artificial neural network library written in Haskell. It’s pretty robust, fast, and expressive, and uses some interesting type level programming techniques which don’t appear to have been used in neural networks before. Grenade does a lot of things right, it’s very expressive in my view, and makes the construction of networks incredibly safe: they essentially can’t crash, and they’ll always be sound. It’s also pretty quick, using BLAS and some hand written C, I’ve found this acceptable for a lot of problems.
or for lispers : http://www.intelligency.org/ai_lisp.php http://www.intelligency.org/ai_lisp.php
or for elixir : https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1 https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1
- keyle 4y agoCould you elaborate why haskell?
- noloblo 4y agoHaskell composing well and type safety makes handling the complex networks of rnns very elegant
- nextos 4y agoIMHO typed FP is important, but not at DL library level. It's going to be hard to match what projects like JAX offer. I think FP, and in particular typed FP, is going to be important for code generation. I think successors of ChatGPT will generate much better code partly because they will explicitly target FP languages. FP code is composable, so it's easier to generate small functions that are correct. Code synthesizers powered by SAT solvers already achieve impressive results, especially when targeting typed FP languages. I imagine that in the not-so-distant future many programmers will be writing type signatures and GPT-like systems will autocomplete function bodies. This might also make formal verification appealing, because one would like to make sure AI-generated code is correct and also because AI-powered assistants will make it more cost-effective.