4 ms·

Deep Learning Reading List

190 points by jmozah 12y ago

therobot24 12y ago
You're going to get most of these from a simple Google search, if you're going to build a list of what to read you should at least put some effort into it. Currently, this list is missing a lot of history behind deep learning - only 3 papers listed! if you want a good set of papers that starts with perceptrons and hebbian learning to multi-layered neural nets and the emergence of what we now refer to as deep networks checkout http://deeplearning.cs.cmu.edu/ http://deeplearning.cs.cmu.edu/
- jmozah 12y ago
  I agree. Thanks for the link. I have added stuff that i started with. will improve this list over time.
  - jmozah 12y ago
    Added the link to the list..
    - amund 12y ago
      Perhaps of interest - http://deeplearning.university http://deeplearning.university - provides a bibliography of recent publications. See also https://news.ycombinator.com/item?id=8334875 https://news.ycombinator.com/item?id=8334875
    - therobot24 12y ago
      Awesome!
ratsimihah 12y ago
Automatic Speech Recognition: a Deep Learning Approach contains an excellent section about deep learning, as well as more content about ASR and hybrid deep learning methods.
- jmozah 12y ago
  can you post the link?
  - ratsimihah 12y ago
    http://www.springer.com/engineering/signals/book/978-1-4471-5778-6 http://www.springer.com/engineering/signals/book/978-1-4471-...
userbinator 12y ago
For those interested in learning deeply about deep learning?
sabalaba 12y ago
I would add "Practical recommendations for gradient-based training of deep architectures" to the list for those who already have a feel for training multi-layer neural nets. It provides a good overview for those that want to learn more about gradient descent, hyperparameter tuning, and other practical considerations involved with training deep architectures. http://arxiv.org/abs/1206.5533 http://arxiv.org/abs/1206.5533
AndrewOMartin 12y ago
Don't forget "Intriguing properties of neural networks", otherwise known as "Does Deep Learning have deep flaws?". In sum, you can teach a network to say "that's a dog" when presented with a picture of a dog, but you'll also be able to (1) find an imperceptibly modified version of the image where it'll say "that's garbage" and (2) intelligently generate an image of noise that also gets recognised as a dog. (1) http://arxiv.org/abs/1312.6199 http://arxiv.org/abs/1312.6199 (2) http://www.newscientist.com/article/dn26691-optical-illusions-fool-computers-into-seeing-things.html http://www.newscientist.com/article/dn26691-optical-illusion...
- HCIdivision17 12y ago
  In (2)'s paper [2], the images are really neat. Staring at it for a few moments, I feel like I can see where the computer is coming from. A bit. The robin, armadillo, centipede, peacock, and bubble all actually have a little swirl of features that - to me at least - resemble the labels provided. But from afar, and you've got to basically damp the noise. I did this by taking my glasses off and leaning about 8 inches from the screen (with the grid taking about 3 inches wide). I've got about -7.5 diopter near-sightedness, so this cleaned the images right up. At least the armadillo I would have guessed, as it's absolutely a little critter walking to the lower right, and it has the demi-circle body and long face. And it might make sense: it was told to make an armadillo, so it did. And nothing else. (I also think the baseball was super clever - if this were abstract art, I would totally have fallen for that classification, as well as a few of the others. There's something cool going on there.) I'm printing and reading the rest of the paper now (about a quarter done). To say it's both fascinating and interesting really understates it. Really neat stuff! [2] http://arxiv.org/pdf/1412.1897v2.pdf http://arxiv.org/pdf/1412.1897v2.pdf
- deleted 12y ago
  [deleted]
sushirain 12y ago
Karpathy's JavaScript demos are the first thing that I would show to a newcomer: http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/convnetjs/
Deckmoresi 12y ago
Deep learning videos and lectures are good. Great list.

190 points by jmozah 12y ago

You're going to get most of these from a simple Google search, if you're going to build a list of what to read you should at least put some effort into it. Currently, this list is missing a lot of history behind deep learning - only 3 papers listed! if you want a good set of papers that starts with perceptrons and hebbian learning to multi-layered neural nets and the emergence of what we now refer to as deep networks checkout http://deeplearning.cs.cmu.edu/ http://deeplearning.cs.cmu.edu/

jmozah 12y ago
I agree. Thanks for the link. I have added stuff that i started with. will improve this list over time.
- jmozah 12y ago
  Added the link to the list..
  - amund 12y ago
    Perhaps of interest - http://deeplearning.university http://deeplearning.university - provides a bibliography of recent publications. See also https://news.ycombinator.com/item?id=8334875 https://news.ycombinator.com/item?id=8334875
  - therobot24 12y ago
    Awesome!

ratsimihah 12y ago

Automatic Speech Recognition: a Deep Learning Approach contains an excellent section about deep learning, as well as more content about ASR and hybrid deep learning methods.

jmozah 12y ago
can you post the link?
- ratsimihah 12y ago
  http://www.springer.com/engineering/signals/book/978-1-4471-5778-6 http://www.springer.com/engineering/signals/book/978-1-4471-...

userbinator 12y ago

For those interested in learning deeply about deep learning?

sabalaba 12y ago

I would add "Practical recommendations for gradient-based training of deep architectures" to the list for those who already have a feel for training multi-layer neural nets. It provides a good overview for those that want to learn more about gradient descent, hyperparameter tuning, and other practical considerations involved with training deep architectures. http://arxiv.org/abs/1206.5533 http://arxiv.org/abs/1206.5533

AndrewOMartin 12y ago

Don't forget "Intriguing properties of neural networks", otherwise known as "Does Deep Learning have deep flaws?". In sum, you can teach a network to say "that's a dog" when presented with a picture of a dog, but you'll also be able to (1) find an imperceptibly modified version of the image where it'll say "that's garbage" and (2) intelligently generate an image of noise that also gets recognised as a dog. (1) http://arxiv.org/abs/1312.6199 http://arxiv.org/abs/1312.6199 (2) http://www.newscientist.com/article/dn26691-optical-illusions-fool-computers-into-seeing-things.html http://www.newscientist.com/article/dn26691-optical-illusion...

HCIdivision17 12y ago
In (2)'s paper [2], the images are really neat. Staring at it for a few moments, I feel like I can see where the computer is coming from. A bit. The robin, armadillo, centipede, peacock, and bubble all actually have a little swirl of features that - to me at least - resemble the labels provided. But from afar, and you've got to basically damp the noise. I did this by taking my glasses off and leaning about 8 inches from the screen (with the grid taking about 3 inches wide). I've got about -7.5 diopter near-sightedness, so this cleaned the images right up. At least the armadillo I would have guessed, as it's absolutely a little critter walking to the lower right, and it has the demi-circle body and long face. And it might make sense: it was told to make an armadillo, so it did. And nothing else. (I also think the baseball was super clever - if this were abstract art, I would totally have fallen for that classification, as well as a few of the others. There's something cool going on there.) I'm printing and reading the rest of the paper now (about a quarter done). To say it's both fascinating and interesting really understates it. Really neat stuff! [2] http://arxiv.org/pdf/1412.1897v2.pdf http://arxiv.org/pdf/1412.1897v2.pdf
deleted 12y ago
[deleted]

sushirain 12y ago

Karpathy's JavaScript demos are the first thing that I would show to a newcomer: http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/convnetjs/

Deckmoresi 12y ago

Deep learning videos and lectures are good. Great list.