6 ms·

Practical Deep Learning for Coders

213 points by rspivak 8y ago

adamnemecek 8y ago
Has anyone here actually built an ml product? A product people use?
- minimaxir 8y ago
  That's setting the goalposts a bit narrow in terms of practical applications of ML. You can apply deep learning to solve problems aren't public facing, such as analytics forecasting. Additionally, a lot of public ML products don't use, and don't need to use, deep learning (e.g. NLP applications).
  - friday99 8y ago
    Analytics forecasting doesn't really seem like a good candidate for deep learning. There are plenty of established methods for forecasting that are simpler, more robust and generally more effective in terms of effort to reward.
- tejaswiy 8y ago
  How do you mean? As a side project? I've seen and built several ML products that are used widely inside Amazon atleast.
- hikarudo 8y ago
  I have built computer vision products at my startup. I was familiar with some machine learning before, but only really learned to apply it on the job. Fast forward 7 years and the initial product and others that followed have been very successful. Machine learning is only a part of the whole product though.
- boia01 8y ago
  Google Photos. It has great search over collections of pictures. You can search for pictures of dogs, sunsets, trees, even pictures of store receipts (which is super convenient for filing expense reports).
  - abraham_lincoln 8y ago
    iOS photos does this natively in search, too.
collyw 8y ago
Ok, from my understanding deep learning is actuially a fairly complicated subject, a bit more than learning the JavaScript framework of the month. So is there any point in treating it as such? Just another skill we should all learn superficially, so that we have a lot of very shallow knowledge?
- DrNuke 8y ago
  You still need a business or research case and then a dataset as any other skill out there, don’t you?
- twtw 8y ago
  > actually a fairly complicated subject This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD). If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort. After that, getting it to work is mostly an empirical and experimental endeavor. (This refers to deep learning as used in practice, not some of the current research topics).
  - AstralStorm 8y ago
    Sometimes, deep learning is too simple to actually work. (complex games with hidden state, exploration, generating art, decision making etc.) Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.) Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.
- minimaxir 8y ago
  It depends on your goal. If you want to make advancements in the field and find innovations that no one else has done (e.g. GANs and Reinforcement Learning), then a superstrong mathematical background is helpful. But if you just want to find the best output given a set of inputs and tune things for business needs, modern tooling like sklearn and TensorFlow/Keras is more than sufficient, and following a set of modern data transformation and model heuristics can get you 80% of the way there. Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.
joshgel 8y ago
I took both of these courses and recommend. Practical is the key word in their description that differentiates them from other courses. The focus is on getting you up and running ASAP, then explaining the details and variation, then finally introducing you to learning how to translate the newest papers into usable code. You're not going to get a PhD from the course, but it will get you started on real-life projects.
- jszymborski 8y ago
  You might get an MSc, however :P Half of my master's thesis was based on practical understanding and skills I learned from the first Fast.ai course. It's also inspired me to start a PhD in a more quantitative field to persue probabilistic models in medicine. (Here's that's master's thesis if you're interested :) Warning, big PDF ahead: https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski_Joseph_Experimental_Medicine_thesis.pdf https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski...)
  - alexcnwy 8y ago
    Same here! Still busy with my MSc thesis though because I'm too busy applying some of the other stuff I learned from Fast.ai on other projects ':)
- abraham_lincoln 8y ago
  How long did it take you to complete each course? Do you have a recommended 'schedule'? I have tried twice, but have not finished.
  - poppingtonic 8y ago
    Try a schedule that keeps you on ~ two hours a day, focusing on at least one of the videos in the 14-video series. So: 14 weeks, paying attention to one video a week and whatever peripheral studies you choose to follow up on based on the ideas Jeremy pushes in his lectures. Edit: also, find datasets online that fit in your interests. There's no better motivation than to solve problems in a domain that you're already familiar with. I grew up on a farm and built a classifier to detect diseases as part of my fast.ai course-work: https://t.me/shambadoctorbot https://t.me/shambadoctorbot
avinassh 8y ago
What are the math prerequisites? Couldn't find anything in about page [0], all it says is: > We assume that everyone taking this course has at least one year of coding experience. so, no math background required? Would really appreciate an input from someone who has done the course [0] - http://course.fast.ai/about.html http://course.fast.ai/about.html
- leesec 8y ago
  I forget where but it used to say roughly a high school level of math understanding. You don't have to do a lot of math in the program but it can be helpful for understanding the concepts. I am very poor at math ( never did much math past high school and that ended in 2009 ), and was able to work through the course, though the course in general is not easy. I really recommend 3Blue1Brown youtube series on linear algebra if you are struggling with some of the math concepts. They have great explanations and great visualizations.
- pmulv 8y ago
  There is a Computational Linear Algebra class taught by one of the founders of fast.ai, but I don't think it's a prerequisite necessarily [0]. I've been meaning to take it - I can't comment on its quality. [0] https://github.com/fastai/numerical-linear-algebra/blob/master/README.md https://github.com/fastai/numerical-linear-algebra/blob/mast...
- jacurtis 8y ago
  Maybe just start watching it if you are interested in learning, instead of trying to put yourself in a box that you might not have the math skills needed for it. It is free, you have nothing to lose. If the topic interests you than I am sure you will figure it out. You can always google a math concept that confuses you. There isn't a test and no one will even know.
  - pmulv 8y ago
    > It is free Just as a warning for anyone price-sensitive interested in taking this course, its cost is non-zero due to cloud costs. It is free to audit though! I'm not trying to dissuade anyone; it is just good to know going into it.
thwy12321 8y ago
Fast.ai uses HN as their own personal marketing platform
kevmo 8y ago
Some of the best instruction I've ever received (and I have too many degrees). These courses are intensely practical. Making sure people understand the theoretical underpinnings comes second to making sure people can actually use DL/ML software for their own purposes. The site could use a reorganization, though. Jeremy and Rachel have created a number of courses, but you have to hunt through their YouTube and fast.ai forum comments to find all of them. So much good content should not be buried!
mooneater 8y ago
> we teach "top down" rather than "bottom up". For instance, you'll learn how to use deep learning to solve your problems in week 1, but will only start to learn why it works in week 2 That sounds more like bottom up to me (practice before theory).
- scabarott 8y ago
  I think they mean it in the sense that: Theoretical foundations --> practical problems = Bottom Up, practice without theory --> building up the theoretical legs to stand on == Top Down
master_yoda_1 8y ago
I don’t understand the big deal about state of the art result. If you know the algorithm (from paper) and the high level tool is available (pytorch) then it is not difficult. If you find it difficult then you are not a good enough software developer and you are not able to produce anything substantial. With fast ai you would mostly end up burning energy in gpu and waste your time. I would better learn c++ then deep learning.
geekfactor 8y ago
If you're looking for a group to take this course with (Jeremy recommends it [1]), we just started a Fall session of the TWiML Online Meetup's Fast.ai study group. More info here: http://twimlai.com/fastai http://twimlai.com/fastai [1] https://twitter.com/jeremyphoward/status/996445183456690176 https://twitter.com/jeremyphoward/status/996445183456690176

213 points by rspivak 8y ago

adamnemecek 8y ago

Has anyone here actually built an ml product? A product people use?

minimaxir 8y ago
That's setting the goalposts a bit narrow in terms of practical applications of ML. You can apply deep learning to solve problems aren't public facing, such as analytics forecasting. Additionally, a lot of public ML products don't use, and don't need to use, deep learning (e.g. NLP applications).
- friday99 8y ago
  Analytics forecasting doesn't really seem like a good candidate for deep learning. There are plenty of established methods for forecasting that are simpler, more robust and generally more effective in terms of effort to reward.
tejaswiy 8y ago
How do you mean? As a side project? I've seen and built several ML products that are used widely inside Amazon atleast.
hikarudo 8y ago
I have built computer vision products at my startup. I was familiar with some machine learning before, but only really learned to apply it on the job. Fast forward 7 years and the initial product and others that followed have been very successful. Machine learning is only a part of the whole product though.
boia01 8y ago
Google Photos. It has great search over collections of pictures. You can search for pictures of dogs, sunsets, trees, even pictures of store receipts (which is super convenient for filing expense reports).
- abraham_lincoln 8y ago
  iOS photos does this natively in search, too.

collyw 8y ago

Ok, from my understanding deep learning is actuially a fairly complicated subject, a bit more than learning the JavaScript framework of the month. So is there any point in treating it as such? Just another skill we should all learn superficially, so that we have a lot of very shallow knowledge?

DrNuke 8y ago
You still need a business or research case and then a dataset as any other skill out there, don’t you?
twtw 8y ago
> actually a fairly complicated subject This is heavily dependent on your background. Deep learning is essentially the simplest thing that could possibly work (parameterized nonlinear function + SGD). If you have experience with calculus roughly equivalent to an undergrad Calc 3, which is required for most fields of engineering, the core ideas of deep learning are very accessible to you without much effort. After that, getting it to work is mostly an empirical and experimental endeavor. (This refers to deep learning as used in practice, not some of the current research topics).
- AstralStorm 8y ago
  Sometimes, deep learning is too simple to actually work. (complex games with hidden state, exploration, generating art, decision making etc.) Those use cases are in constant flux and research. (Though teams like DeepMind are making some serious inroads.) Other problem is that DL is still highly sample inefficient - it needs many more examples than you'd want or sometimes even have to bootstrap.
minimaxir 8y ago
It depends on your goal. If you want to make advancements in the field and find innovations that no one else has done (e.g. GANs and Reinforcement Learning), then a superstrong mathematical background is helpful. But if you just want to find the best output given a set of inputs and tune things for business needs, modern tooling like sklearn and TensorFlow/Keras is more than sufficient, and following a set of modern data transformation and model heuristics can get you 80% of the way there. Medium thought-pieces/YouTubers conflate both perspectives, which has become a problem.

joshgel 8y ago

I took both of these courses and recommend. Practical is the key word in their description that differentiates them from other courses. The focus is on getting you up and running ASAP, then explaining the details and variation, then finally introducing you to learning how to translate the newest papers into usable code. You're not going to get a PhD from the course, but it will get you started on real-life projects.

jszymborski 8y ago
You might get an MSc, however :P Half of my master's thesis was based on practical understanding and skills I learned from the first Fast.ai course. It's also inspired me to start a PhD in a more quantitative field to persue probabilistic models in medicine. (Here's that's master's thesis if you're interested :) Warning, big PDF ahead: https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski_Joseph_Experimental_Medicine_thesis.pdf https://www.cs.mcgill.ca/~jszymb/thesis/260528685_Szymborski...)
- alexcnwy 8y ago
  Same here! Still busy with my MSc thesis though because I'm too busy applying some of the other stuff I learned from Fast.ai on other projects ':)
abraham_lincoln 8y ago
How long did it take you to complete each course? Do you have a recommended 'schedule'? I have tried twice, but have not finished.
- poppingtonic 8y ago
  Try a schedule that keeps you on ~ two hours a day, focusing on at least one of the videos in the 14-video series. So: 14 weeks, paying attention to one video a week and whatever peripheral studies you choose to follow up on based on the ideas Jeremy pushes in his lectures. Edit: also, find datasets online that fit in your interests. There's no better motivation than to solve problems in a domain that you're already familiar with. I grew up on a farm and built a classifier to detect diseases as part of my fast.ai course-work: https://t.me/shambadoctorbot https://t.me/shambadoctorbot

avinassh 8y ago

What are the math prerequisites? Couldn't find anything in about page [0], all it says is: > We assume that everyone taking this course has at least one year of coding experience. so, no math background required? Would really appreciate an input from someone who has done the course [0] - http://course.fast.ai/about.html http://course.fast.ai/about.html

leesec 8y ago
I forget where but it used to say roughly a high school level of math understanding. You don't have to do a lot of math in the program but it can be helpful for understanding the concepts. I am very poor at math ( never did much math past high school and that ended in 2009 ), and was able to work through the course, though the course in general is not easy. I really recommend 3Blue1Brown youtube series on linear algebra if you are struggling with some of the math concepts. They have great explanations and great visualizations.
pmulv 8y ago
There is a Computational Linear Algebra class taught by one of the founders of fast.ai, but I don't think it's a prerequisite necessarily [0]. I've been meaning to take it - I can't comment on its quality. [0] https://github.com/fastai/numerical-linear-algebra/blob/master/README.md https://github.com/fastai/numerical-linear-algebra/blob/mast...
jacurtis 8y ago
Maybe just start watching it if you are interested in learning, instead of trying to put yourself in a box that you might not have the math skills needed for it. It is free, you have nothing to lose. If the topic interests you than I am sure you will figure it out. You can always google a math concept that confuses you. There isn't a test and no one will even know.
- pmulv 8y ago
  > It is free Just as a warning for anyone price-sensitive interested in taking this course, its cost is non-zero due to cloud costs. It is free to audit though! I'm not trying to dissuade anyone; it is just good to know going into it.

thwy12321 8y ago

Fast.ai uses HN as their own personal marketing platform

kevmo 8y ago

Some of the best instruction I've ever received (and I have too many degrees). These courses are intensely practical. Making sure people understand the theoretical underpinnings comes second to making sure people can actually use DL/ML software for their own purposes. The site could use a reorganization, though. Jeremy and Rachel have created a number of courses, but you have to hunt through their YouTube and fast.ai forum comments to find all of them. So much good content should not be buried!

mooneater 8y ago

> we teach "top down" rather than "bottom up". For instance, you'll learn how to use deep learning to solve your problems in week 1, but will only start to learn why it works in week 2 That sounds more like bottom up to me (practice before theory).

scabarott 8y ago
I think they mean it in the sense that: Theoretical foundations --> practical problems = Bottom Up, practice without theory --> building up the theoretical legs to stand on == Top Down

master_yoda_1 8y ago

I don’t understand the big deal about state of the art result. If you know the algorithm (from paper) and the high level tool is available (pytorch) then it is not difficult. If you find it difficult then you are not a good enough software developer and you are not able to produce anything substantial. With fast ai you would mostly end up burning energy in gpu and waste your time. I would better learn c++ then deep learning.

geekfactor 8y ago

If you're looking for a group to take this course with (Jeremy recommends it [1]), we just started a Fall session of the TWiML Online Meetup's Fast.ai study group. More info here: http://twimlai.com/fastai http://twimlai.com/fastai [1] https://twitter.com/jeremyphoward/status/996445183456690176 https://twitter.com/jeremyphoward/status/996445183456690176