7 ms·

Deep Learning Courses

159 points by cjdulberger 11y ago

ojbyrne 11y ago
So I see a link at the bottom: "Andrew Ng's Coursera course provides a good introduction to deep learning" which links to his "Machine Learning" class. This leads me to believe that "deep learning" is a synonym for "machine learning." Honest question: is that the case? Just a rebranding?
- crazypyro 11y ago
  Speaking of their academic origins, deep learning is a subset of machine learning. It just also happens to be the most exciting and the one with the biggest publicity/results in recent years, so its often (mis)-labeled to describe the entire machine learning field. Andrew Ng's course still contains fundamental knowledge necessary to understand the motivations and reasoning behind deep learning (along with a lot of background ML knowledge that isn't needed), though, so I think its a good resource to link.
- hellofunk 11y ago
  Machine learning is a broad field of techniques; deep learning is an area of machine learning that I think generally focuses on neural network techniques, and usually rather large networks at that. Machine learning in isolation doesn't necessarily mean neural network techniques are used at all. Nor does deep learning exclusively mean neural networks, but usually "deep networks" are a part of it.
- dave_sullivan 11y ago
  Deep learning is anything that's been written about neural networks since 2006. Seriously. Everything else is some subset of "machine learning" of which "deep learning" is also a subset.
- paulsutter 11y ago
  Deep learning is machine learning where the system develops ("learns") its own featureset. Generally by a multilayer neural network. Selecting features is typically the aspect of machine learning that required the most domain experience, and in deep learning that part is done by an algorithm. The features can be layered. For example, speech recognition could have layers something like phonemes, morphemes, words, concepts, etc. building such featursets by hand is challenging, in deep learning the system learns useful features by detecting patterns. Interesting that the most challenging technical task is among the first to be replaced by an algorithm ;)
- bunderbunder 11y ago
  To the extent that it's a new umbrella term that mostly encompasses a set of techniques that have been around for a couple decades now, yeah, it's just a rebranding. But I think the rebranding makes some sense because it calls attention to a common characteristic shared by all the techniques that fall under the brand: They tend to learn their own feature transformations, which is cool because it means you don't have to put nearly so much effort into figuring out how to curate the input.
- _delirium 11y ago
  To get an idea of a machine-learning course not focused on deep learning, compare Andrew Ng's other course, the one he teaches at Stanford, which is exclusively focused on other aspects of ML: http://cs229.stanford.edu/schedule.html http://cs229.stanford.edu/schedule.html
- SilasX 11y ago
  Yeah, it would be more accurate to say that Ng's course (as best I remember it) provides a good introduction to machine learning and covers unsupervised learning, for which deep learning is one approach.
- vonnik 11y ago
  "Deep learning" is used a couple ways. In the loose sense employed by some journalists, deep learning is probably synonymous with both machine learning and AI. It's at the cutting edge of the current hype. In the strict sense, deep learning refers to neural networks with more than one hidden layer. The depth of neural networks is equal to the longest path between input and output nodes. "Shallow" networks like a simple autoencoder might have two layers. They're not considered deep or part of deep learning. But if you stack them together, you have a deep net; e.g. many restricted Boltzmann machines form a deep-belief network. [1] As @paulsutter mentioned, one aspect of having several hidden, or intermediate, layers in a neural network is that you can combine relatively simple, granular features (like individual pixels or words) into more complex combinations. Neural networks recombine simple features automatically, and then learn which groupings should be lent significance as signals through the backpropagation of error. They're attracting all this hype because they actually do something amazing, albeit through brute force computation. Many in AI scoff at neural networks because they've been around a long time and have no particular elegance, but we're now in a historical moment where we have the hardware to make them work, and they're breaking records in almost every data type; e.g. images, sound, time series, etc. So no, it's not a rebranding, it's a thing. We can now replicate the human faculty of perception with machines in many domains, and that's going to make the future quite weird. [1] http://deeplearning4j.org/restrictedboltzmannmachine.html http://deeplearning4j.org/restrictedboltzmannmachine.html
- nabla9 11y ago
  Deep learning (or deep machine learning) methods are representation-learning methods with multiple levels of representations.
ericmo 11y ago
Glad to know there are recordings of this. I registered, but I've been missing the lectures because of timezone differences.
ris 11y ago
Translation: CUDA indoctrination courses.
- itsnotlupus 11y ago
  Is indoctrination even needed at this stage? Are there openCL equivalents to the popular GPU-accelerated NN frameworks/libraries? You only need to convince people when they have another choice.
  - agibsonccc 11y ago
    Disclaimer, it's my project, but I run an open source project called deeplearning4j, who's algorithms have a hardware abstraction layer built in to them called nd4j. You get numpy on the jvm and hardware as a jar file. Deeplearning4j itself is built on top of that. Would love to help spread deep learning to different runtimes. Here's the current work being done on opencl: https://github.com/deeplearning4j/nd4j/tree/master/nd4j-jocl-parent https://github.com/deeplearning4j/nd4j/tree/master/nd4j-jocl... We'd love to get this finished. Bit more to do yet though...definitely looking for contributors here. You'll get opencl neural nets for free.
    - dharma1 11y ago
      interesting. What's the performance like on the same/equivalent GPU when comparing CUDA to OpenCL?
      - agibsonccc 11y ago
        Need to run empirical benchmarks. CUDA is usually faster. I'd like to run my own benchmarks with nd4j though. We have our own benchmark setup that works for every backend. It allows us to do some interesting things. Cuda itself is usually faster with data transfer latency though[1]. Looking forward to running these ourselves after our opencl support kicks in (only the kernels are written =/) I plan on basing the work for open cl on our cuda work which is fairly well established at this point (mainly doing optimizations not much change in architecture) [1]: http://arxiv.org/pdf/1005.2581.pdf http://arxiv.org/pdf/1005.2581.pdf
krat0sprakhar 11y ago
Few other Machine Learning / Deep Learning courses here - https://github.com/prakhar1989/awesome-courses#machine-learning https://github.com/prakhar1989/awesome-courses#machine-learn... </shameless-plug>

159 points by cjdulberger 11y ago

So I see a link at the bottom: "Andrew Ng's Coursera course provides a good introduction to deep learning" which links to his "Machine Learning" class. This leads me to believe that "deep learning" is a synonym for "machine learning." Honest question: is that the case? Just a rebranding?

crazypyro 11y ago
Speaking of their academic origins, deep learning is a subset of machine learning. It just also happens to be the most exciting and the one with the biggest publicity/results in recent years, so its often (mis)-labeled to describe the entire machine learning field. Andrew Ng's course still contains fundamental knowledge necessary to understand the motivations and reasoning behind deep learning (along with a lot of background ML knowledge that isn't needed), though, so I think its a good resource to link.
hellofunk 11y ago
Machine learning is a broad field of techniques; deep learning is an area of machine learning that I think generally focuses on neural network techniques, and usually rather large networks at that. Machine learning in isolation doesn't necessarily mean neural network techniques are used at all. Nor does deep learning exclusively mean neural networks, but usually "deep networks" are a part of it.
dave_sullivan 11y ago
Deep learning is anything that's been written about neural networks since 2006. Seriously. Everything else is some subset of "machine learning" of which "deep learning" is also a subset.
paulsutter 11y ago
Deep learning is machine learning where the system develops ("learns") its own featureset. Generally by a multilayer neural network. Selecting features is typically the aspect of machine learning that required the most domain experience, and in deep learning that part is done by an algorithm. The features can be layered. For example, speech recognition could have layers something like phonemes, morphemes, words, concepts, etc. building such featursets by hand is challenging, in deep learning the system learns useful features by detecting patterns. Interesting that the most challenging technical task is among the first to be replaced by an algorithm ;)
bunderbunder 11y ago
To the extent that it's a new umbrella term that mostly encompasses a set of techniques that have been around for a couple decades now, yeah, it's just a rebranding. But I think the rebranding makes some sense because it calls attention to a common characteristic shared by all the techniques that fall under the brand: They tend to learn their own feature transformations, which is cool because it means you don't have to put nearly so much effort into figuring out how to curate the input.
_delirium 11y ago
To get an idea of a machine-learning course not focused on deep learning, compare Andrew Ng's other course, the one he teaches at Stanford, which is exclusively focused on other aspects of ML: http://cs229.stanford.edu/schedule.html http://cs229.stanford.edu/schedule.html
SilasX 11y ago
Yeah, it would be more accurate to say that Ng's course (as best I remember it) provides a good introduction to machine learning and covers unsupervised learning, for which deep learning is one approach.
vonnik 11y ago
"Deep learning" is used a couple ways. In the loose sense employed by some journalists, deep learning is probably synonymous with both machine learning and AI. It's at the cutting edge of the current hype. In the strict sense, deep learning refers to neural networks with more than one hidden layer. The depth of neural networks is equal to the longest path between input and output nodes. "Shallow" networks like a simple autoencoder might have two layers. They're not considered deep or part of deep learning. But if you stack them together, you have a deep net; e.g. many restricted Boltzmann machines form a deep-belief network. [1] As @paulsutter mentioned, one aspect of having several hidden, or intermediate, layers in a neural network is that you can combine relatively simple, granular features (like individual pixels or words) into more complex combinations. Neural networks recombine simple features automatically, and then learn which groupings should be lent significance as signals through the backpropagation of error. They're attracting all this hype because they actually do something amazing, albeit through brute force computation. Many in AI scoff at neural networks because they've been around a long time and have no particular elegance, but we're now in a historical moment where we have the hardware to make them work, and they're breaking records in almost every data type; e.g. images, sound, time series, etc. So no, it's not a rebranding, it's a thing. We can now replicate the human faculty of perception with machines in many domains, and that's going to make the future quite weird. [1] http://deeplearning4j.org/restrictedboltzmannmachine.html http://deeplearning4j.org/restrictedboltzmannmachine.html
nabla9 11y ago
Deep learning (or deep machine learning) methods are representation-learning methods with multiple levels of representations.

ericmo 11y ago

Glad to know there are recordings of this. I registered, but I've been missing the lectures because of timezone differences.

ris 11y ago

Translation: CUDA indoctrination courses.

itsnotlupus 11y ago
Is indoctrination even needed at this stage? Are there openCL equivalents to the popular GPU-accelerated NN frameworks/libraries? You only need to convince people when they have another choice.
- agibsonccc 11y ago
  Disclaimer, it's my project, but I run an open source project called deeplearning4j, who's algorithms have a hardware abstraction layer built in to them called nd4j. You get numpy on the jvm and hardware as a jar file. Deeplearning4j itself is built on top of that. Would love to help spread deep learning to different runtimes. Here's the current work being done on opencl: https://github.com/deeplearning4j/nd4j/tree/master/nd4j-jocl-parent https://github.com/deeplearning4j/nd4j/tree/master/nd4j-jocl... We'd love to get this finished. Bit more to do yet though...definitely looking for contributors here. You'll get opencl neural nets for free.
  - dharma1 11y ago
    interesting. What's the performance like on the same/equivalent GPU when comparing CUDA to OpenCL?
    - agibsonccc 11y ago
      Need to run empirical benchmarks. CUDA is usually faster. I'd like to run my own benchmarks with nd4j though. We have our own benchmark setup that works for every backend. It allows us to do some interesting things. Cuda itself is usually faster with data transfer latency though[1]. Looking forward to running these ourselves after our opencl support kicks in (only the kernels are written =/) I plan on basing the work for open cl on our cuda work which is fairly well established at this point (mainly doing optimizations not much change in architecture) [1]: http://arxiv.org/pdf/1005.2581.pdf http://arxiv.org/pdf/1005.2581.pdf

krat0sprakhar 11y ago

Few other Machine Learning / Deep Learning courses here - https://github.com/prakhar1989/awesome-courses#machine-learning https://github.com/prakhar1989/awesome-courses#machine-learn... </shameless-plug>