7 ms·

Deep Learning: An MIT Press book in preparation

224 points by mutor 12y ago

jbarrow 12y ago
This looks like it has the potential to be a great resource! Not to mention it's coming out of one of the big name schools of deep learning (along with UToronto and Stanford). I imagine, though, that anyone not well versed in college mathematics may have issues with the explanations. If you want a good introductory resource, but either haven't covered or have forgotten some of the math in this book, I would recommend one of two resources: [1] MetaCademy: http://metacademy.org http://metacademy.org [2] Neural Networks and Deep Learning (In Progress): http://neuralnetworksanddeeplearning.com http://neuralnetworksanddeeplearning.com The first will take you through all the math first through some online courses and textbooks, and the second is a good general purpose introduction that I recommend to anybody interested in neural nets.
justin66 12y ago
Pages look ok in Firefox, borderline unreadable in Chrome. I wonder what that's about.
- thomasfoster96 12y ago
  They've converted a PDF to be shown in HTML. Possibly using PDF.js or something.
  - felixr 12y ago
    Looks like they used pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlEX https://github.com/coolwanglu/pdf2htmlEX)
plg 12y ago
this is great, I hope the authors keep a .pdf version available for dl once its done
nfoz 12y ago
Why is "deep learning" so hype nowadays? They seem like just another AI tool that will be prone to over-fitting datasets and provide an analysis that is difficult to mathematically characterize and understand in a reliable way. Not here to be cynical/negative -- they might be of great value, this is not my expertise. Can someone explain why deep learning articles are receiving attention rather than, say, Support Vector Machines / kernel-based methods of pattern analysis? Or other nonlinear analysis? Are they related?
- marcodena 12y ago
  http://www.slideshare.net/hammawan/deep-neural-networks http://www.slideshare.net/hammawan/deep-neural-networks :)
- shock-value 12y ago
  I think you are right in your sentiment that deep learning is becoming over-hyped. I've written before that it's mostly a method of brute forcing the problem of AI that happens to work very efficiently on modern GPUs and other parallel processing units. That being said, for the tasks it does well on (mostly facial and other types of simple image recognition), it does extremely well compared to everything else out there. (And it may very well end up as a part of a future architecture that better simulates general intelligence.) However, I don't think it is reasonable to assume that AI tools can only be valuable if they can be rigorously mathematically characterized. The "holy grail" of intelligence -- that is, actual human intelligence -- certainly can't or at least hasn't been mathematically characterized, and I don't think anyone will ever offer any "proof" (mathematical or otherwise) that the biological brain is inherently wired to arrive at or at least tend to converge to correct solutions to intelligence problems. And of course no one will argue that human intelligence isn't valuable!
  - eli_gottlieb 12y ago
    If my machine learning algorithms went wrong as regularly as the term "human error" denotes, I wouldn't be able to use them to run much of anything. Identifying faces, for instance, is just a matter of statistical accuracy, of pattern-matching, but a more complex task like automated asset-trading or world domination requires a program that will make fewer mistakes than even many trained humans.
vonnik 12y ago
I hear O'Reilly's coming out with a book soon as well.
bentoner 12y ago
See also Michael Nielsen's book Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/
- ajays 12y ago
  Unfortunately, the chapter(s) on Deep Learning haven't even been written yet.
iandanforth 12y ago
Hrm ... there has got to be a better format for soliciting edits. Github, Google Doc, ???
discardorama 12y ago
Sigh. The authors have taken a PDF and generated HTML out of it. I can understand the desire to maintain copyright, but this is ridiculous.
gamesbrainiac 12y ago
First it was data mining, then machine learning, and now deep learning. What's the next name? Clairvoyant learning?
- chestervonwinch 12y ago
  The latter is a subset of the former and refers to particular architectures/strategies for doing machine learning/data mining. Think deep as in hierarchies not deep as in "whoa, man". Some ideas in deep learning are new; some are old. It's not entirely a rebranding.
fizixer 12y ago
Question: Does anyone have updated information about Hinton's 'trillion parameter network' progress (http://deeplearning.net/2013/05/20/g-hinton-wants-to-train-neural-nets-with-trillions-of-parameters/ http://deeplearning.net/2013/05/20/g-hinton-wants-to-train-n...) from almost a year and a half ago? It's been quite a while and even Ng has demonstrated that a billion parameter setup could be built for $20k using commodity hardware (https://news.ycombinator.com/item?id=5896684 https://news.ycombinator.com/item?id=5896684). I wonder what's happening at Google labs as of August 2014.
shepardrtc 12y ago
Here's a talk by Geoffrey Hinton back in 2007 about Deep Learning neural networks: https://www.youtube.com/watch?v=AyzOUbkUf3M https://www.youtube.com/watch?v=AyzOUbkUf3M

224 points by mutor 12y ago

This looks like it has the potential to be a great resource! Not to mention it's coming out of one of the big name schools of deep learning (along with UToronto and Stanford). I imagine, though, that anyone not well versed in college mathematics may have issues with the explanations. If you want a good introductory resource, but either haven't covered or have forgotten some of the math in this book, I would recommend one of two resources: [1] MetaCademy: http://metacademy.org http://metacademy.org [2] Neural Networks and Deep Learning (In Progress): http://neuralnetworksanddeeplearning.com http://neuralnetworksanddeeplearning.com The first will take you through all the math first through some online courses and textbooks, and the second is a good general purpose introduction that I recommend to anybody interested in neural nets.

justin66 12y ago

Pages look ok in Firefox, borderline unreadable in Chrome. I wonder what that's about.

thomasfoster96 12y ago
They've converted a PDF to be shown in HTML. Possibly using PDF.js or something.
- felixr 12y ago
  Looks like they used pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlEX https://github.com/coolwanglu/pdf2htmlEX)

plg 12y ago

this is great, I hope the authors keep a .pdf version available for dl once its done

nfoz 12y ago

Why is "deep learning" so hype nowadays? They seem like just another AI tool that will be prone to over-fitting datasets and provide an analysis that is difficult to mathematically characterize and understand in a reliable way. Not here to be cynical/negative -- they might be of great value, this is not my expertise. Can someone explain why deep learning articles are receiving attention rather than, say, Support Vector Machines / kernel-based methods of pattern analysis? Or other nonlinear analysis? Are they related?

marcodena 12y ago
http://www.slideshare.net/hammawan/deep-neural-networks http://www.slideshare.net/hammawan/deep-neural-networks :)
shock-value 12y ago
I think you are right in your sentiment that deep learning is becoming over-hyped. I've written before that it's mostly a method of brute forcing the problem of AI that happens to work very efficiently on modern GPUs and other parallel processing units. That being said, for the tasks it does well on (mostly facial and other types of simple image recognition), it does extremely well compared to everything else out there. (And it may very well end up as a part of a future architecture that better simulates general intelligence.) However, I don't think it is reasonable to assume that AI tools can only be valuable if they can be rigorously mathematically characterized. The "holy grail" of intelligence -- that is, actual human intelligence -- certainly can't or at least hasn't been mathematically characterized, and I don't think anyone will ever offer any "proof" (mathematical or otherwise) that the biological brain is inherently wired to arrive at or at least tend to converge to correct solutions to intelligence problems. And of course no one will argue that human intelligence isn't valuable!
- eli_gottlieb 12y ago
  If my machine learning algorithms went wrong as regularly as the term "human error" denotes, I wouldn't be able to use them to run much of anything. Identifying faces, for instance, is just a matter of statistical accuracy, of pattern-matching, but a more complex task like automated asset-trading or world domination requires a program that will make fewer mistakes than even many trained humans.

vonnik 12y ago

I hear O'Reilly's coming out with a book soon as well.

bentoner 12y ago

See also Michael Nielsen's book Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/

ajays 12y ago
Unfortunately, the chapter(s) on Deep Learning haven't even been written yet.

iandanforth 12y ago

Hrm ... there has got to be a better format for soliciting edits. Github, Google Doc, ???

discardorama 12y ago

Sigh. The authors have taken a PDF and generated HTML out of it. I can understand the desire to maintain copyright, but this is ridiculous.

gamesbrainiac 12y ago

First it was data mining, then machine learning, and now deep learning. What's the next name? Clairvoyant learning?

chestervonwinch 12y ago
The latter is a subset of the former and refers to particular architectures/strategies for doing machine learning/data mining. Think deep as in hierarchies not deep as in "whoa, man". Some ideas in deep learning are new; some are old. It's not entirely a rebranding.

fizixer 12y ago

Question: Does anyone have updated information about Hinton's 'trillion parameter network' progress (http://deeplearning.net/2013/05/20/g-hinton-wants-to-train-neural-nets-with-trillions-of-parameters/ http://deeplearning.net/2013/05/20/g-hinton-wants-to-train-n...) from almost a year and a half ago? It's been quite a while and even Ng has demonstrated that a billion parameter setup could be built for $20k using commodity hardware (https://news.ycombinator.com/item?id=5896684 https://news.ycombinator.com/item?id=5896684). I wonder what's happening at Google labs as of August 2014.

shepardrtc 12y ago

Here's a talk by Geoffrey Hinton back in 2007 about Deep Learning neural networks: https://www.youtube.com/watch?v=AyzOUbkUf3M https://www.youtube.com/watch?v=AyzOUbkUf3M