3 ms·

Ok, from my understanding deep learning is actuially a fairly complicated subject, a bit more than learning the JavaScript framework of the month. So is there

by collyw 8y ago

Ok, from my understanding deep learning is actuially a fairly complicated subject, a bit more than learning the JavaScript framework of the month. So is there any point in treating it as such? Just another skill we should all learn superficially, so that we have a lot of very shallow knowledge?

DrNuke 8y ago
You still need a business or research case and then a dataset as any other skill out there, don’t you?
twtw 8y ago
> actually a fairly complicated subject This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD). If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort. After that, getting it to work is mostly an empirical and experimental endeavor. (This refers to deep learning as used in practice, not some of the current research topics).
- AstralStorm 8y ago
  Sometimes, deep learning is too simple to actually work. (complex games with hidden state, exploration, generating art, decision making etc.) Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.) Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.
minimaxir 8y ago
It depends on your goal. If you want to make advancements in the field and find innovations that no one else has done (e.g. GANs and Reinforcement Learning), then a superstrong mathematical background is helpful. But if you just want to find the best output given a set of inputs and tune things for business needs, modern tooling like sklearn and TensorFlow/Keras is more than sufficient, and following a set of modern data transformation and model heuristics can get you 80% of the way there. Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.
- fizwhiz 8y ago
  cough siraj raval cough
leesec 8y ago
I've taken the first part of this (not the newest version using Pytorch, tho I'd like to retake someday soon), and found this class to be extremely useful. I do agree that in order to be optimally efficient you'd want to have a background understanding of statistics/linear algebra; however, I love this class because it gets you up and running very quickly and excited about the content. It's also my understanding that the ML stack is making it more and more accessible to actually apply these algorithms. Sure, after you take this class you probably won't go out and invent new algorithmic breakthroughs, but many people in this class do participate in in Kaggle competitions solving real world problems (some of the assignments are submissions) and do very well, even winning occasionally. The point of the class is to generate interest and to get people useful quickly (make neural nets uncool again), it'd be much harder to get the field to blossom if you required people to start with years of stats/math beforehand.
zeroxfe 8y ago
I think that just like most people don't need to know about the extremely complex internals of databases or networks, we should get deep-learning systems to a point where users can treat them the same way. (Of course there's always value in knowing the details -- it helps to strive towards lowering the barrier to entry into complex systems.)
- collyw 8y ago
  I wish people would learn to use relational databases properly. There is a lot of crap out in the wild.
dragonwriter 8y ago
The advantage of learning lots of “deep” skills superficially is that it gives you the perspective to have a better idea of which ones you should dive more deeply into to address particular real-world problems. (It may even give you the tools to apply some of them to the real world problems they are most simply applicable to without further study, as a bonus.)
jacurtis 8y ago
They promise to teach you deep learning, not to master it. For many people, this is a good primer into basic concepts and how deep learning works from a high level. If they are still interested then they can work towards mastery. But you have to start somewhere.
nafizh 8y ago
This course is certainly not superficial. I have taken it as an international fellow. I suggest you do the first two lectures to get a feel for it.

by collyw 8y ago

DrNuke 8y ago

You still need a business or research case and then a dataset as any other skill out there, don’t you?

twtw 8y ago

> actually a fairly complicated subject This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD). If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort. After that, getting it to work is mostly an empirical and experimental endeavor. (This refers to deep learning as used in practice, not some of the current research topics).

AstralStorm 8y ago
Sometimes, deep learning is too simple to actually work. (complex games with hidden state, exploration, generating art, decision making etc.) Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.) Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.

minimaxir 8y ago

It depends on your goal. If you want to make advancements in the field and find innovations that no one else has done (e.g. GANs and Reinforcement Learning), then a superstrong mathematical background is helpful. But if you just want to find the best output given a set of inputs and tune things for business needs, modern tooling like sklearn and TensorFlow/Keras is more than sufficient, and following a set of modern data transformation and model heuristics can get you 80% of the way there. Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.

fizwhiz 8y ago
cough siraj raval cough

leesec 8y ago

I've taken the first part of this (not the newest version using Pytorch, tho I'd like to retake someday soon), and found this class to be extremely useful. I do agree that in order to be optimally efficient you'd want to have a background understanding of statistics/linear algebra; however, I love this class because it gets you up and running very quickly and excited about the content. It's also my understanding that the ML stack is making it more and more accessible to actually apply these algorithms. Sure, after you take this class you probably won't go out and invent new algorithmic breakthroughs, but many people in this class do participate in in Kaggle competitions solving real world problems (some of the assignments are submissions) and do very well, even winning occasionally. The point of the class is to generate interest and to get people useful quickly (make neural nets uncool again), it'd be much harder to get the field to blossom if you required people to start with years of stats/math beforehand.

zeroxfe 8y ago

I think that just like most people don't need to know about the extremely complex internals of databases or networks, we should get deep-learning systems to a point where users can treat them the same way. (Of course there's always value in knowing the details -- it helps to strive towards lowering the barrier to entry into complex systems.)

collyw 8y ago
I wish people would learn to use relational databases properly. There is a lot of crap out in the wild.

dragonwriter 8y ago

The advantage of learning lots of “deep” skills superficially is that it gives you the perspective to have a better idea of which ones you should dive more deeply into to address particular real-world problems. (It may even give you the tools to apply some of them to the real world problems they are most simply applicable to without further study, as a bonus.)

jacurtis 8y ago

They promise to teach you deep learning, not to master it. For many people, this is a good primer into basic concepts and how deep learning works from a high level. If they are still interested then they can work towards mastery. But you have to start somewhere.

nafizh 8y ago

This course is certainly not superficial. I have taken it as an international fellow. I suggest you do the first two lectures to get a feel for it.