5 ms·

MIT 6.S191: Introduction to Deep Learning

softwarejosh 7y ago
cool im gonna take this thanks
melling 7y ago
There are a few years of this course. I list them here: https://github.com/melling/MathAndScienceNotes/blob/master/machine_learning.md https://github.com/melling/MathAndScienceNotes/blob/master/m... Direct link to 2018-2019: http://www.youtube.com/watch?v=5v1JnYv_yWs&list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI http://www.youtube.com/watch?v=5v1JnYv_yWs&list=PLtBw6njQRU-... 2017: https://www.youtube.com/playlist?list=PLkkuNyzb8LmxFutYuPA7B4oiMn6cjD6Rs https://www.youtube.com/playlist?list=PLkkuNyzb8LmxFutYuPA7B...
caycep 7y ago
This may be slightly OT, but my layperson perspective is for a pre-eminent STEM university, that MIT always seems to lag behind its rivals in deep learning and the latest ml advances were spearheaded at Stanford/Silicon Valley outfits, the Montreal/Toronto academic communities, Caltech and to some extent in the UK. Whereas its background should have made it one of the leaders in the field. Am I overlooking something, or just gross misinformed? Did MIT culturally stay married for a while to the Minsky school of deterministic AI vs. the Big Data statistical approach?
- dwrodri 7y ago
  Source/Disclaimer: Am graduate student in Boston area who works in Computer Engineering. I am certain I am overlooking some important info that others should add. You're right in Saying that Stanford, UCB, UW, and other schools have had a much stronger track record producing deep learning models that have revolutionized certain fields (img classification, segmentation, nlp, etc.). I think MIT has hired some seriously talented researchers who have chosen to invest their time into different avenues of research that align more closely with the "engineering" side of deep learning. For example, Eyeriss[1] kickstarted the AI accelerator race. Halide[2] is a DSL+runtime used as the basis in a lot of deep learning compilation tools, like Tensor Comprehensions[3]. I don't think you're grossly misinformed, I just think that while other schools have invested in the theory, MIT is betting on a level of abstraction that's a bit lower. 1 = http://eyeriss.mit.edu/ http://eyeriss.mit.edu/ 2 = https://halide-lang.org/ https://halide-lang.org/ 3 = https://research.fb.com/blog/2018/02/announcing-tensor-comprehensions/ https://research.fb.com/blog/2018/02/announcing-tensor-compr... 4 = https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/
- lonelappde 7y ago
  Big Data requires Big Computers (and Big Brother), which came from Google. MIT did't have the $100Billion to spend on hardware and create the culture.
- mrdrozdov 7y ago
  Publications at NeurIPS/ICML is a reasonable proxy for contribution to the field. https://medium.com/@chuvpilo/ai-research-rankings-2019-insights-from-neurips-and-icml-leading-ai-conferences-ee6953152c1a https://medium.com/@chuvpilo/ai-research-rankings-2019-insig... In 2019, the top 5 institutions were: 1. Google (USA) — 167.3 2. Stanford University (USA) — 82.3 3. MIT (USA) — 69.8 4. Carnegie Mellon University (USA) — 67.7 5. UC Berkeley (USA) — 54.0
  - etrk 7y ago
    Citations tend to be a better indicator of impact. There are some interesting analysis here [1]. MIT has the most citations, and they've also published the most papers by far. If considering citations per paper, MIT falls in the middle of the pack. Toronto is an outlier with a very high number of citations per paper. NeurIPs is a big conference covering lots of topics, though. All of this says relatively little about MIT's true impact on deep learning in the last eight years. [1] https://www.microsoft.com/en-us/research/project/academic/articles/neurips-conference-analytics/ https://www.microsoft.com/en-us/research/project/academic/ar...
drongoking 7y ago
Any idea on how this compares with the other deep learning intro courses available?
_coveredInBees 7y ago
Looks neat. Personally bummed that it goes with Tensorflow, though I guess that may be related to the course being sponsored in part by Google. Pretty much all the latest research is being published in Pytorch and even OpenAI switched to Pytorch recently.
- ganeshkrishnan 7y ago
  any courses like this that teach pytorch? I am thinking of taking up this course and would like to give pytorch a shot too
  - _coveredInBees 7y ago
    CS231n from Stanford uses Pytorch
  - jestinjoy1 7y ago
    This one looks good. https://atcold.github.io/pytorch-Deep-Learning/?fbclid=IwAR2eOdot67TmKenYUtAkMP0UNnz8WYFjmGorouGX5FYGc9FBlFDW9svhsqg https://atcold.github.io/pytorch-Deep-Learning/?fbclid=IwAR2...
- deleted 7y ago
  [deleted]
- Tenoke 7y ago
  Both PyTorch and Tensorflow are used extensively. If you look at the last year's most popular models - GPT2 and StyleGan - they are both Tensorflow. When I check on Scholar there are many, many results for both Tensorflow and PyTorch from 2019. The most recent stuff I've been working on (because that's what they were published in) have been in Jax. PyTorch is one of the biggest frameworks but it's unreasonable to suggest it is the only one that matters.
  - _coveredInBees 7y ago
    GPT-2 was developed by Open-AI. They recently switched to Pytorch themselves for similar reasons [1]. I didn't claim that all research that matters is happening on Pytorch, but it is very true that most non-Google research is [2]. Having used both, Pytorch has been a far better experience for me, and lately, most of the reference implementations I care about are released in Pytorch, so it makes it very easy to experiment with and also pull into my own code base if needed. Also, for an intro class, Pytorch would have been a better choice imo, because it has a simpler and cleaner API. In my experience, it was a lot easier to get up and running with Pytorch compared to Tensorflow. 1 - https://openai.com/blog/openai-pytorch/ https://openai.com/blog/openai-pytorch/ 2 - https://news.ycombinator.com/item?id=21216200 https://news.ycombinator.com/item?id=21216200
ksharifbd 7y ago
Great! Will check that out.
bor100003 7y ago
Sorry for the layman's questions but what's the difference between this course and Lex Fridman's ?

cool im gonna take this thanks

There are a few years of this course. I list them here: https://github.com/melling/MathAndScienceNotes/blob/master/machine_learning.md https://github.com/melling/MathAndScienceNotes/blob/master/m... Direct link to 2018-2019: http://www.youtube.com/watch?v=5v1JnYv_yWs&list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI http://www.youtube.com/watch?v=5v1JnYv_yWs&list=PLtBw6njQRU-... 2017: https://www.youtube.com/playlist?list=PLkkuNyzb8LmxFutYuPA7B4oiMn6cjD6Rs https://www.youtube.com/playlist?list=PLkkuNyzb8LmxFutYuPA7B...

caycep 7y ago

This may be slightly OT, but my layperson perspective is for a pre-eminent STEM university, that MIT always seems to lag behind its rivals in deep learning and the latest ml advances were spearheaded at Stanford/Silicon Valley outfits, the Montreal/Toronto academic communities, Caltech and to some extent in the UK. Whereas its background should have made it one of the leaders in the field. Am I overlooking something, or just gross misinformed? Did MIT culturally stay married for a while to the Minsky school of deterministic AI vs. the Big Data statistical approach?

dwrodri 7y ago
Source/Disclaimer: Am graduate student in Boston area who works in Computer Engineering. I am certain I am overlooking some important info that others should add. You're right in Saying that Stanford, UCB, UW, and other schools have had a much stronger track record producing deep learning models that have revolutionized certain fields (img classification, segmentation, nlp, etc.). I think MIT has hired some seriously talented researchers who have chosen to invest their time into different avenues of research that align more closely with the "engineering" side of deep learning. For example, Eyeriss[1] kickstarted the AI accelerator race. Halide[2] is a DSL+runtime used as the basis in a lot of deep learning compilation tools, like Tensor Comprehensions[3]. I don't think you're grossly misinformed, I just think that while other schools have invested in the theory, MIT is betting on a level of abstraction that's a bit lower. 1 = http://eyeriss.mit.edu/ http://eyeriss.mit.edu/ 2 = https://halide-lang.org/ https://halide-lang.org/ 3 = https://research.fb.com/blog/2018/02/announcing-tensor-comprehensions/ https://research.fb.com/blog/2018/02/announcing-tensor-compr... 4 = https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/
lonelappde 7y ago
Big Data requires Big Computers (and Big Brother), which came from Google. MIT did't have the $100Billion to spend on hardware and create the culture.
mrdrozdov 7y ago
Publications at NeurIPS/ICML is a reasonable proxy for contribution to the field. https://medium.com/@chuvpilo/ai-research-rankings-2019-insights-from-neurips-and-icml-leading-ai-conferences-ee6953152c1a https://medium.com/@chuvpilo/ai-research-rankings-2019-insig... In 2019, the top 5 institutions were: 1. Google (USA) — 167.3 2. Stanford University (USA) — 82.3 3. MIT (USA) — 69.8 4. Carnegie Mellon University (USA) — 67.7 5. UC Berkeley (USA) — 54.0
- etrk 7y ago
  Citations tend to be a better indicator of impact. There are some interesting analysis here [1]. MIT has the most citations, and they've also published the most papers by far. If considering citations per paper, MIT falls in the middle of the pack. Toronto is an outlier with a very high number of citations per paper. NeurIPs is a big conference covering lots of topics, though. All of this says relatively little about MIT's true impact on deep learning in the last eight years. [1] https://www.microsoft.com/en-us/research/project/academic/articles/neurips-conference-analytics/ https://www.microsoft.com/en-us/research/project/academic/ar...

drongoking 7y ago

Any idea on how this compares with the other deep learning intro courses available?

_coveredInBees 7y ago

Looks neat. Personally bummed that it goes with Tensorflow, though I guess that may be related to the course being sponsored in part by Google. Pretty much all the latest research is being published in Pytorch and even OpenAI switched to Pytorch recently.

ganeshkrishnan 7y ago
any courses like this that teach pytorch? I am thinking of taking up this course and would like to give pytorch a shot too
- _coveredInBees 7y ago
  CS231n from Stanford uses Pytorch
- jestinjoy1 7y ago
  This one looks good. https://atcold.github.io/pytorch-Deep-Learning/?fbclid=IwAR2eOdot67TmKenYUtAkMP0UNnz8WYFjmGorouGX5FYGc9FBlFDW9svhsqg https://atcold.github.io/pytorch-Deep-Learning/?fbclid=IwAR2...
deleted 7y ago
[deleted]
Tenoke 7y ago
Both PyTorch and Tensorflow are used extensively. If you look at the last year's most popular models - GPT2 and StyleGan - they are both Tensorflow. When I check on Scholar there are many, many results for both Tensorflow and PyTorch from 2019. The most recent stuff I've been working on (because that's what they were published in) have been in Jax. PyTorch is one of the biggest frameworks but it's unreasonable to suggest it is the only one that matters.
- _coveredInBees 7y ago
  GPT-2 was developed by Open-AI. They recently switched to Pytorch themselves for similar reasons [1]. I didn't claim that all research that matters is happening on Pytorch, but it is very true that most non-Google research is [2]. Having used both, Pytorch has been a far better experience for me, and lately, most of the reference implementations I care about are released in Pytorch, so it makes it very easy to experiment with and also pull into my own code base if needed. Also, for an intro class, Pytorch would have been a better choice imo, because it has a simpler and cleaner API. In my experience, it was a lot easier to get up and running with Pytorch compared to Tensorflow. 1 - https://openai.com/blog/openai-pytorch/ https://openai.com/blog/openai-pytorch/ 2 - https://news.ycombinator.com/item?id=21216200 https://news.ycombinator.com/item?id=21216200

ksharifbd 7y ago

Great! Will check that out.

bor100003 7y ago

Sorry for the layman's questions but what's the difference between this course and Lex Fridman's ?