4 ms·

Not sure neural net expert Yann LeCun (yann.lecun.com) would agree with your neural networks comment. Neural nets have taken on a life of their own since the ti

by T_S_ 16y ago

Not sure neural net expert Yann LeCun (yann.lecun.com) would agree with your neural networks comment. Neural nets have taken on a life of their own since the time they were originated and actually have little direct connection with neuroscience these days.

alextp 16y ago
This is true, but the very good deep learning results are very recent (from less than 5 years ago), and they generally require very different models and/or training strategies from the standard multilayer perceptron you find in most AI textbooks. So it's an omission, but I'm not aware of any textbooks that cover these new neural networks in depth, as basic research in this area is still going in a frantic pace.
- Groxx 16y ago
  Yeah, I'm finding that there's little book-format info on deep neural networks as well. Similarly for belief networks. Gotta hunt down some journals, and wield some Google-fu.
  - tudorachim 16y ago
    The most promising approaches for neural nets these days involve learning generative models using Restricted Boltzmann Machines (RBMs). Geoffrey Hinton at UToronto has some good seminal papers on these, like http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf. See http://www.cs.toronto.edu/~hinton/ http://www.cs.toronto.edu/~hinton/ for excellent videos of digit generation / classification. As a note, this approach outperforms all others (sans tweaks), whereas neural nets used to be beaten by SVMs and the like.
    - alextp 16y ago
      Actually, the best recent results in this area involve discriminatively trained stacked auto-encoders using second-order algorithms (for which the code, as of Sept 2010, is still private to the group at the university of toronto), as described in James Martens's paper at this year's ICML: http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pd...

by T_S_ 16y ago

alextp 16y ago

This is true, but the very good deep learning results are very recent (from less than 5 years ago), and they generally require very different models and/or training strategies from the standard multilayer perceptron you find in most AI textbooks. So it's an omission, but I'm not aware of any textbooks that cover these new neural networks in depth, as basic research in this area is still going in a frantic pace.

Groxx 16y ago
Yeah, I'm finding that there's little book-format info on deep neural networks as well. Similarly for belief networks. Gotta hunt down some journals, and wield some Google-fu.
- tudorachim 16y ago
  The most promising approaches for neural nets these days involve learning generative models using Restricted Boltzmann Machines (RBMs). Geoffrey Hinton at UToronto has some good seminal papers on these, like http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf. See http://www.cs.toronto.edu/~hinton/ http://www.cs.toronto.edu/~hinton/ for excellent videos of digit generation / classification. As a note, this approach outperforms all others (sans tweaks), whereas neural nets used to be beaten by SVMs and the like.
  - alextp 16y ago
    Actually, the best recent results in this area involve discriminatively trained stacked auto-encoders using second-order algorithms (for which the code, as of Sept 2010, is still private to the group at the university of toronto), as described in James Martens's paper at this year's ICML: http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pd...