6 ms·

MIT lecture series on deep learning in January 2020

221 points by melling 7y ago

deleted 7y ago
[deleted]
chrispeel 7y ago
The site loads very slowly. If you're impatient, videos [1] on Youtube from Lex Fridman seem to be a big part of the course. [1] https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOk...
vo2maxer 7y ago
Alternative link: http://introtodeeplearning.com/ http://introtodeeplearning.com/
ummonk 7y ago
Wow, even an MIT subdomain gets the hug of death?
- kick 7y ago
  A lot of MIT subdomains are hosted by students. A MIT subdomain getting enough traffic to bring it down isn't surprising in itself. It's that this one is hosted by the administration itself (seemingly) that makes it notable.
dr_kiszonka 7y ago
Look exciting. Realistically, what can one expect to learn from the class? Would I learn any practical skills or is it just an overview?
- rckoepke 7y ago
  Looks like overview to me. Guys like Francois Chollet aren't going to spend their time doing an in-depth tutorial, and even if he spent the whole 90-minute lecture walking you through the specific thing he invented (Keras), it probably wouldn't be enough time to get a naive programmer up to competency. Each week is a different lecturer.
nelson687 7y ago
Is this an online course?
nothis 7y ago
I feel like there's about 1000 hours of high quality AI lectures available for free on the internet and while I do believe in a certain amount of selflessness in education, I am skeptical that any of that is providing more than a glimpse of what you need to know to be productive at it. In other words, there's a thousand hours of material out there, which probably takes 10000 hours to actually get into so you might as well put all that effort into properly studying it at a university, getting the knowledge not covered in the lectures alone and a degree to prove it in the end. If you're just "curious" about AI, a really good half hour lecture should get you up to speed.
- ptd 7y ago
  Why do you have to go to university to properly study machine learning?
- thatcat 7y ago
  Studying it at a university means moving and having a certain background. While i agree that these resources are hard to break down into a curriculum, theres nothing stopping you from copying a university curriculum at home and doing work on your own...
  - diffeomorphism 7y ago
    > theres nothing stopping you Yet most people don't. Same for learning an instrument, carpentering etc.. While you could in principle self-study lots of things, study groups, structure, people to talk to and discuss with and even just "we meet every Thursday at noon to ..." are not negligible.
    - rckoepke 7y ago
      > study groups, structure, people to talk to and discuss with and even just "we meet every Thursday at noon to ..." are not negligible. Indeed. These resources are stupendously valuable, and probably somewhat easier to find/generate in some areas than most others.
- 7y ago
stevespang 7y ago
The MIT link "times out" . . .
waiseristy 7y ago
So Lex Fridman curated this page and is the subject of many videos within it. Seems like Missy Cummings and Filip Piekniewski's assumption that MIT has scrubbed him from the site is unfounded. [1] https://twitter.com/missy_cummings/status/1179497003635662850 https://twitter.com/missy_cummings/status/117949700363566285... [2] https://blog.piekniewski.info/2019/11/18/late2019-the-wizards-of-oz/ https://blog.piekniewski.info/2019/11/18/late2019-the-wizard... Lex's blocking of dissenting opinions on twitter is still egregious though.
Vervious 7y ago
Notably, these are all winter courses, taught over the course of three weeks in January. They don't have the formality and institutional support of a semester-long course. Often they are taught by students. MIT has a ton of people working in this area, and I'm not sure this particular group of people is representative.
Mugwort 7y ago
I'm definitely going to listen to the Lex Fridman lectures simply because I enjoyed his interview with Judea Pearl so much.
codingslave 7y ago
The gap that I see in current machine learning is that everyone is learning how to use the popular models, but no one knows how to construct a new model that solves a new problem. So everyone can download word vectors and use them for what they're good at, but the second you get off the beaten track, almost all machine learning practitioners fall flat. I really dont think this is due to how new the field is, rather that very few people have the mathematical maturity and experience to actually use optimization theory and linear algebra to construct new highly specific language models. There is simply no information available about doing this. You can learn about the underpinnings of matrix factorization and how that relates to word vectors, take that further and read about eigen vectors, but still, its too thin.
- M5x7wI3CmbEem10 7y ago
  wouldn’t this just require a few textbooks in linear/non-linear optimization, mathematical modeling, and real analysis?
  - codingslave 7y ago
    I think its way different than that, those would just be precursors, and in cases like real analysis, superfluous. Instead it would look something like, I have a Universal Sentence Encoder architecture, but its not performing well on my data, aside from tweaking the training set, how can I take this architecture and change it to work better with my individual problem? Assume here that the problem one is trying to solve is very similar to the models original use case. The number of people on the planet that can do this successfully, without wasting months of time messing around with tensorflow is extremely small. But this is where the value is. These massive catch all models only work for the people creating them, just jamming them into any NLP model will always produce sub par and probably unusable results
    - rckoepke 7y ago
      I believe Google's AutoML is attempting to answer these types of questions. It's obviously internal-only so others can't fork the research...but it has helped them invent new specific networks like "EfficientNet for EdgeTPU" [0]. I think humans can still invent new macro structures like CNN's...but humans are inherently shit at analyzing "what if we removed one neuron in the 2nd hidden layer?". The subtle tweaking is really best left to an automated recursion process. Humans are better at seeing/inventing macro structures - such as adapting the unidirectional GPT to a bidirectional ELMO/BERT. After the invention, humans are generally pretty good at determining "whether" a network can be used to solve a particular task, although not infallible [1: Can BERT generate sentences from a prompt like GPT?] But computers are once again often better at quickly determining whether which (ELMO, BERT, or GPT) perform better on a particular task for which they are all at least feasibly suited. 0: http://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html http://ai.googleblog.com/2019/08/efficientnet-edgetpu-creati... 1: https://ai.stackexchange.com/questions/9141/can-bert-be-used-for-sentence-generating-tasks https://ai.stackexchange.com/questions/9141/can-bert-be-used...
deleted 7y ago
[deleted]

221 points by melling 7y ago

deleted 7y ago

[deleted]

The site loads very slowly. If you're impatient, videos [1] on Youtube from Lex Fridman seem to be a big part of the course. [1] https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf https://www.youtube.com/playlist?list=PLrAXtmErZgOeiKm4sgNOk...

vo2maxer 7y ago

Alternative link: http://introtodeeplearning.com/ http://introtodeeplearning.com/

ummonk 7y ago

Wow, even an MIT subdomain gets the hug of death?

kick 7y ago
A lot of MIT subdomains are hosted by students. A MIT subdomain getting enough traffic to bring it down isn't surprising in itself. It's that this one is hosted by the administration itself (seemingly) that makes it notable.

dr_kiszonka 7y ago

Look exciting. Realistically, what can one expect to learn from the class? Would I learn any practical skills or is it just an overview?

rckoepke 7y ago
Looks like overview to me. Guys like Francois Chollet aren't going to spend their time doing an in-depth tutorial, and even if he spent the whole 90-minute lecture walking you through the specific thing he invented (Keras), it probably wouldn't be enough time to get a naive programmer up to competency. Each week is a different lecturer.

nelson687 7y ago

Is this an online course?

nothis 7y ago

I feel like there's about 1000 hours of high quality AI lectures available for free on the internet and while I do believe in a certain amount of selflessness in education, I am skeptical that any of that is providing more than a glimpse of what you need to know to be productive at it. In other words, there's a thousand hours of material out there, which probably takes 10000 hours to actually get into so you might as well put all that effort into properly studying it at a university, getting the knowledge not covered in the lectures alone and a degree to prove it in the end. If you're just "curious" about AI, a really good half hour lecture should get you up to speed.

ptd 7y ago
Why do you have to go to university to properly study machine learning?
thatcat 7y ago
Studying it at a university means moving and having a certain background. While i agree that these resources are hard to break down into a curriculum, theres nothing stopping you from copying a university curriculum at home and doing work on your own...
- diffeomorphism 7y ago
  > theres nothing stopping you Yet most people don't. Same for learning an instrument, carpentering etc.. While you could in principle self-study lots of things, study groups, structure, people to talk to and discuss with and even just "we meet every Thursday at noon to ..." are not negligible.
  - rckoepke 7y ago
    > study groups, structure, people to talk to and discuss with and even just "we meet every Thursday at noon to ..." are not negligible. Indeed. These resources are stupendously valuable, and probably somewhat easier to find/generate in some areas than most others.
7y ago

stevespang 7y ago

The MIT link "times out" . . .

waiseristy 7y ago

So Lex Fridman curated this page and is the subject of many videos within it. Seems like Missy Cummings and Filip Piekniewski's assumption that MIT has scrubbed him from the site is unfounded. [1] https://twitter.com/missy_cummings/status/1179497003635662850 https://twitter.com/missy_cummings/status/117949700363566285... [2] https://blog.piekniewski.info/2019/11/18/late2019-the-wizards-of-oz/ https://blog.piekniewski.info/2019/11/18/late2019-the-wizard... Lex's blocking of dissenting opinions on twitter is still egregious though.

Vervious 7y ago

Notably, these are all winter courses, taught over the course of three weeks in January. They don't have the formality and institutional support of a semester-long course. Often they are taught by students. MIT has a ton of people working in this area, and I'm not sure this particular group of people is representative.

Mugwort 7y ago

I'm definitely going to listen to the Lex Fridman lectures simply because I enjoyed his interview with Judea Pearl so much.

codingslave 7y ago

The gap that I see in current machine learning is that everyone is learning how to use the popular models, but no one knows how to construct a new model that solves a new problem. So everyone can download word vectors and use them for what they're good at, but the second you get off the beaten track, almost all machine learning practitioners fall flat. I really dont think this is due to how new the field is, rather that very few people have the mathematical maturity and experience to actually use optimization theory and linear algebra to construct new highly specific language models. There is simply no information available about doing this. You can learn about the underpinnings of matrix factorization and how that relates to word vectors, take that further and read about eigen vectors, but still, its too thin.

M5x7wI3CmbEem10 7y ago
wouldn’t this just require a few textbooks in linear/non-linear optimization, mathematical modeling, and real analysis?
- codingslave 7y ago
  I think its way different than that, those would just be precursors, and in cases like real analysis, superfluous. Instead it would look something like, I have a Universal Sentence Encoder architecture, but its not performing well on my data, aside from tweaking the training set, how can I take this architecture and change it to work better with my individual problem? Assume here that the problem one is trying to solve is very similar to the models original use case. The number of people on the planet that can do this successfully, without wasting months of time messing around with tensorflow is extremely small. But this is where the value is. These massive catch all models only work for the people creating them, just jamming them into any NLP model will always produce sub par and probably unusable results
  - rckoepke 7y ago
    I believe Google's AutoML is attempting to answer these types of questions. It's obviously internal-only so others can't fork the research...but it has helped them invent new specific networks like "EfficientNet for EdgeTPU" [0]. I think humans can still invent new macro structures like CNN's...but humans are inherently shit at analyzing "what if we removed one neuron in the 2nd hidden layer?". The subtle tweaking is really best left to an automated recursion process. Humans are better at seeing/inventing macro structures - such as adapting the unidirectional GPT to a bidirectional ELMO/BERT. After the invention, humans are generally pretty good at determining "whether" a network can be used to solve a particular task, although not infallible [1: Can BERT generate sentences from a prompt like GPT?] But computers are once again often better at quickly determining whether which (ELMO, BERT, or GPT) perform better on a particular task for which they are all at least feasibly suited. 0: http://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html http://ai.googleblog.com/2019/08/efficientnet-edgetpu-creati... 1: https://ai.stackexchange.com/questions/9141/can-bert-be-used-for-sentence-generating-tasks https://ai.stackexchange.com/questions/9141/can-bert-be-used...

deleted 7y ago

[deleted]