4 ms·

I know of 4 projects for deep learning based on Theano. Keras, Blocks and Lasagne all seem to share the same goal of being more libraries than framework. You c

by jre 12y ago

I know of 4 projects for deep learning based on Theano. Keras, Blocks and Lasagne all seem to share the same goal of being more libraries than framework. You can use only one part (e.g. a Layer implementation, training algo) without having to pull in everything : https://github.com/bartvm/blocks https://github.com/bartvm/blocks https://github.com/benanne/Lasagne https://github.com/benanne/Lasagne Then there is pylearn2, which look more like a framework and seems to be a good candidate for becoming the GPU-accelerated scikit-learn : https://github.com/lisa-lab/pylearn2 https://github.com/lisa-lab/pylearn2 I have started using blocks and did some tests with pylearn2. Anybody with more experience want to share the strength/weaknesses of each of those projects ?

albertzeyer 12y ago
I know some more. Some of them are made as libraries, some are just code examples, where you however could extract out some relevant code. * PyLearn, LISA labs, http://deeplearning.net/software/pylearn2/ http://deeplearning.net/software/pylearn2/ * LSTM, http://deeplearning.net/tutorial/lstm.html#lstm http://deeplearning.net/tutorial/lstm.html#lstm * LSTM, https://github.com/skaae/nntools https://github.com/skaae/nntools * LSTM, https://github.com/JonathanRaiman/theano_lstm https://github.com/JonathanRaiman/theano_lstm * LSTM, https://github.com/mohammadpz/Recurrent-Neural-Networks https://github.com/mohammadpz/Recurrent-Neural-Networks * LSTM, http://christianherta.de/lehre/dataScience/machineLearning/neuralNetworks/LSTM.php http://christianherta.de/lehre/dataScience/machineLearning/n... * LSTM, https://gist.github.com/jpuigcerver/9358036 https://gist.github.com/jpuigcerver/9358036 * LSTM + CTC, https://github.com/kastnerkyle/net https://github.com/kastnerkyle/net * Speech modeling, LSTM, https://github.com/kastnerkyle/speech_density https://github.com/kastnerkyle/speech_density * FF + RNN, https://github.com/lmjohns3/theano-nets https://github.com/lmjohns3/theano-nets * FF + RNN, https://github.com/lmjohns3/theanets https://github.com/lmjohns3/theanets Speech: https://github.com/lmjohns3/arrnn-experiment/blob/master/tasks/speech.py https://github.com/lmjohns3/arrnn-experiment/blob/master/tas... * FF, https://github.com/benanne/Lasagne https://github.com/benanne/Lasagne * RNN, https://github.com/pascanur/trainingRNNs https://github.com/pascanur/trainingRNNs * RNN, https://github.com/pascanur/GroundHog https://github.com/pascanur/GroundHog (Razvan Pascanu, KyungHyun Cho, Caglar Gulcehre) * RNN + CTC, https://github.com/shawntan/rnn-experiment https://github.com/shawntan/rnn-experiment (Shawn Tan) * RNN + CTC, https://github.com/shawntan/theano-ctc https://github.com/shawntan/theano-ctc (Shawn Tan) * RNN + CTC, https://github.com/rakeshvar/rnn_ctc https://github.com/rakeshvar/rnn_ctc * RNN + CTC, OCR, https://github.com/rakeshvar/chamanti_ocr https://github.com/rakeshvar/chamanti_ocr, https://github.com/rakeshvar/chamanti3_ocr https://github.com/rakeshvar/chamanti3_ocr * RNN, https://github.com/gwtaylor/theano-rnn https://github.com/gwtaylor/theano-rnn * LSTM, RBM, DBN, https://github.com/kratarth1203/NeuralNet https://github.com/kratarth1203/NeuralNet * RBM, https://github.com/benanne/morb https://github.com/benanne/morb * Q-learning, https://github.com/spragunr/deep_q_rl https://github.com/spragunr/deep_q_rl * Deep Generative Models, https://github.com/dpkingma/nips14-ssl https://github.com/dpkingma/nips14-ssl * RNN, agents, “bricks”: https://github.com/bartvm/blocks https://github.com/bartvm/blocks * NTM, https://github.com/shawntan/neural-turing-machines/ https://github.com/shawntan/neural-turing-machines/ * RL + CNN, https://github.com/brian473/neural_rl https://github.com/brian473/neural_rl * DRAW RNN, https://github.com/jbornschein/draw https://github.com/jbornschein/draw And this is far from complete, there are countless more examples. Just search on GitHub. I just filtered out the ones which interest me (which at least have RNNs/LSTMs or some other interesting things).
- benanne 12y ago
  Nice work! Since you mentioned you're looking for RNNs/LSTMs specifically: the implementation at https://github.com/skaae/nntools https://github.com/skaae/nntools is an extension of Lasagne (which used to be called nntools) and will be merged into the library at some point. Hopefully in time for the first release, but we don't know yet if that will be feasible.
- elarosca 12y ago
  I would also like to mention my project: pydeeplearn. You can find it here https://github.com/mihaelacr/pydeeplearn https://github.com/mihaelacr/pydeeplearn. I think it's main advantage is that it uses theano under the hood but the user does not need to know theano at all. The most complete implementations are those of RBMs and DBNs, but I also have CNNs. The library has support for adversarial training, as presented in the paper "Explaining and Harnessing Adversarial Examples" by Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy. I recently also integrated spearmint into the library, so hyperparameter optimization comes in for free.
dave_sullivan 12y ago
Fwiw, we're using pylearn2 and blocks at Ersatz Labs. Both use Theano. I'd recommend them, particularly if you are into python. It's hard to build a good NN framework: subtle math bugs can creep in, the field is changing quickly, and there are varied opinions on implementation details (some more valid than others). Even just learning a single framework requires a good deal of effort.

by jre 12y ago

albertzeyer 12y ago

I know some more. Some of them are made as libraries, some are just code examples, where you however could extract out some relevant code. * PyLearn, LISA labs, http://deeplearning.net/software/pylearn2/ http://deeplearning.net/software/pylearn2/ * LSTM, http://deeplearning.net/tutorial/lstm.html#lstm http://deeplearning.net/tutorial/lstm.html#lstm * LSTM, https://github.com/skaae/nntools https://github.com/skaae/nntools * LSTM, https://github.com/JonathanRaiman/theano_lstm https://github.com/JonathanRaiman/theano_lstm * LSTM, https://github.com/mohammadpz/Recurrent-Neural-Networks https://github.com/mohammadpz/Recurrent-Neural-Networks * LSTM, http://christianherta.de/lehre/dataScience/machineLearning/neuralNetworks/LSTM.php http://christianherta.de/lehre/dataScience/machineLearning/n... * LSTM, https://gist.github.com/jpuigcerver/9358036 https://gist.github.com/jpuigcerver/9358036 * LSTM + CTC, https://github.com/kastnerkyle/net https://github.com/kastnerkyle/net * Speech modeling, LSTM, https://github.com/kastnerkyle/speech_density https://github.com/kastnerkyle/speech_density * FF + RNN, https://github.com/lmjohns3/theano-nets https://github.com/lmjohns3/theano-nets * FF + RNN, https://github.com/lmjohns3/theanets https://github.com/lmjohns3/theanets Speech: https://github.com/lmjohns3/arrnn-experiment/blob/master/tasks/speech.py https://github.com/lmjohns3/arrnn-experiment/blob/master/tas... * FF, https://github.com/benanne/Lasagne https://github.com/benanne/Lasagne * RNN, https://github.com/pascanur/trainingRNNs https://github.com/pascanur/trainingRNNs * RNN, https://github.com/pascanur/GroundHog https://github.com/pascanur/GroundHog (Razvan Pascanu, KyungHyun Cho, Caglar Gulcehre) * RNN + CTC, https://github.com/shawntan/rnn-experiment https://github.com/shawntan/rnn-experiment (Shawn Tan) * RNN + CTC, https://github.com/shawntan/theano-ctc https://github.com/shawntan/theano-ctc (Shawn Tan) * RNN + CTC, https://github.com/rakeshvar/rnn_ctc https://github.com/rakeshvar/rnn_ctc * RNN + CTC, OCR, https://github.com/rakeshvar/chamanti_ocr https://github.com/rakeshvar/chamanti_ocr, https://github.com/rakeshvar/chamanti3_ocr https://github.com/rakeshvar/chamanti3_ocr * RNN, https://github.com/gwtaylor/theano-rnn https://github.com/gwtaylor/theano-rnn * LSTM, RBM, DBN, https://github.com/kratarth1203/NeuralNet https://github.com/kratarth1203/NeuralNet * RBM, https://github.com/benanne/morb https://github.com/benanne/morb * Q-learning, https://github.com/spragunr/deep_q_rl https://github.com/spragunr/deep_q_rl * Deep Generative Models, https://github.com/dpkingma/nips14-ssl https://github.com/dpkingma/nips14-ssl * RNN, agents, “bricks”: https://github.com/bartvm/blocks https://github.com/bartvm/blocks * NTM, https://github.com/shawntan/neural-turing-machines/ https://github.com/shawntan/neural-turing-machines/ * RL + CNN, https://github.com/brian473/neural_rl https://github.com/brian473/neural_rl * DRAW RNN, https://github.com/jbornschein/draw https://github.com/jbornschein/draw And this is far from complete, there are countless more examples. Just search on GitHub. I just filtered out the ones which interest me (which at least have RNNs/LSTMs or some other interesting things).

benanne 12y ago
Nice work! Since you mentioned you're looking for RNNs/LSTMs specifically: the implementation at https://github.com/skaae/nntools https://github.com/skaae/nntools is an extension of Lasagne (which used to be called nntools) and will be merged into the library at some point. Hopefully in time for the first release, but we don't know yet if that will be feasible.
elarosca 12y ago
I would also like to mention my project: pydeeplearn. You can find it here https://github.com/mihaelacr/pydeeplearn https://github.com/mihaelacr/pydeeplearn. I think it's main advantage is that it uses theano under the hood but the user does not need to know theano at all. The most complete implementations are those of RBMs and DBNs, but I also have CNNs. The library has support for adversarial training, as presented in the paper "Explaining and Harnessing Adversarial Examples" by Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy. I recently also integrated spearmint into the library, so hyperparameter optimization comes in for free.

dave_sullivan 12y ago

Fwiw, we're using pylearn2 and blocks at Ersatz Labs. Both use Theano. I'd recommend them, particularly if you are into python. It's hard to build a good NN framework: subtle math bugs can creep in, the field is changing quickly, and there are varied opinions on implementation details (some more valid than others). Even just learning a single framework requires a good deal of effort.