11 ms·

Practical Deep Learning for Coders 2019

556 points by jph00 8y ago

DTE 8y ago
Congrats, Jeremy and crew!
- jph00 8y ago
  Many thanks. I'm really happy with how it turned out. I hope you all like it! :)
  - input_device 8y ago
    I'm really interested in learning the math basics for deep learning. Is there an online guide that you can point us to if we want to learn the basics of calculus, linear algebra, probability theory? In other words is there a "fast.ai" version of "math for deep learning" out there?
    - joshvm 8y ago
      Yes for linear algebra, though it's probably best suited to people with at least some exposure to matrix/vector math (for example you've used SVD but have no idea how or why it works): https://www.fast.ai/2017/07/17/num-lin-alg/ https://www.fast.ai/2017/07/17/num-lin-alg/ For probability, machine learning is more about statistics (the two are related, but courses explicitly about probability will cover different things), so I would lean towards that. An Introduction to Statistical Learning in R (ISLR) is a frequently recommended book. You can ignore the R and do the exercises in Python. If you actually want to learn about probability, you can look at MIT's course: https://ocw.mit.edu/resources/res-6-012-introduction-to-probability-spring-2018/ https://ocw.mit.edu/resources/res-6-012-introduction-to-prob... EDIT: If you've never been exposed to calculus, many people swear by Khan Academy's videos.
      - input_device 8y ago
        This is great. Thanks you!
        Canadauni 8y ago
        I would also recommend the MIT calculus courses on edX. It has been a great refresher for me since the last I looked at calc was 8 years ago.
        input_device 8y ago
        I'll check it out, thanks!
sigfubar 8y ago
Does your definition of "coders" include those of us with at most middle school mathematics knowledge? I've tried some machine learning courses before, but couldn't make sense of the vast majority of information.
- jph00 8y ago
  Yes it'll be a bit more work for you, but it should be OK. I provide links to more math background where required in the course.
  - mywrathacademia 8y ago
    Can the exercises in the fast.ai deep learning course be done in another language other than python? And how much time in weeks should it take to complete the course? If my laptop doesn't have a GPU, can I still do all exercises?
    - rasmus1610 8y ago
      The whole course builds upon the fastai library written in python so there is no point in not using python. But I feel like you can gain a pretty good understanding of deep learning in general by watching the videos. The live course was held over a time of 7 weeks. It should take around 70h to finish it in total but I really depends on how deep you want to dive in and how easy it is for you to understand.
    - cr0sh 8y ago
      > If my laptop doesn't have a GPU, can I still do all exercises? To answer this, while technically you could do the exercises, it is possible that it will take longer (perhaps much longer) to train models and whatnot without a GPU. Where you'll really have a hard time will be anything for deep learning and large neural networks. What might take a few seconds or minutes to train on a GPU might take hours with just a CPU with a few cores. But not to worry - in the course it explains that you can use a variety of "cloud based" GPU systems that are fairly cheap to access (provided you have a credit card). So if you can't get access to a GPU, there are other ways that won't put you in the poorhouse (just make absolutely sure to completely shutdown and/or delete your VM instances you spin up). Also - while you could use another language, it is best to stick with Python for now, because it has become almost the defacto language for ML and Deep Learning, at least as far as for education and such. Mainly because it has such great libraries for scientific computing available, plus it is easily approachable. It also has a great interface to Tensorflow (and ultimately CUDA and NVidia GPUs - also, get used to the fact that if you do anything DL related, you will be using NVidia almost exclusively; while AMD and others have their own DL hardware, it isn't nearly as well supported - maybe it will be better supported in the future). That said - Tensorflow does have decent support in a number of languages, especially C++ - so if you are more familiar with that language, you'll be set to do some amazing things once you complete the course. But Python, again, is really where it's at. Tensorflow is a great library that abstracts away a lot of the pain in dealing with CUDA. But even it has pain - and there are other libraries layered over it that make things even easier to work with (ie - Keras). Seriously; in the training I've had with DL (mainly thru Udacity), we had to learn how to build a simplified DL "library" of methods for building simple neural networks. Once you learn about Tensorflow, you will have an epiphany of "OMG so much easier!". Then - when you learn about Keras, it's yet another epiphany - it makes Tensorflow look "difficult". Of course, it's best to learn in this manner if you can, so that you understand how the lower levels work - instead of everything being a black-box. Of course, at some level there will be "black-boxes"; for instance, you may or may not want to learn how Tensorflow interfaces with CUDA. Or how CUDA interfaces with the GPU. Or certain other methods in Tensorflow or some other library that perhaps wraps certain methods and functionality. Honestly, things are much nicer today than they were even 5 years ago. You can decide just what level of depth you want to explore down to.
knicholes 8y ago
So we should watch these newer ones instead of the older ones?
- jph00 8y ago
  For sure. It's a giant step over last year's course.
  - jonahx 8y ago
    This is awesome, many thanks. Jeremy, I know you're a fellow fan of J -- is it possible to do modern ML with J? It seems like numpy is just a worse version of the array paradigm from J/APL, but I'm guessing all the specialized ML packages are the real value of python here? Any idea of anyone has done similar work in J?
  - amch 8y ago
    Has the classic ML course been deprecated? I noticed that following link no longer works: https://course.fast.ai/lessonsml1/lesson1.html https://course.fast.ai/lessonsml1/lesson1.html EDIT: looks like they've been moved to the following link instead: http://course18.fast.ai/ml.html http://course18.fast.ai/ml.html
  - alienreborn 8y ago
    Is there a way to create lifestyle config for Sagemaker manually instead of using fast.ai created IAM ARN and config?
  - gmartres 8y ago
    What are the main differences with last year's course ? Also can we expect http://course18.fast.ai/part2.html http://course18.fast.ai/part2.html to be updated in the near future ?
2bitencryption 8y ago
I'm so excited to dive into this. One question: is there a reason you opted for PyTorch over Keras? I had the impression that Keras was the go-to for "easy as ABC" neural networking. EDIT: bonus question! I'm really fascinated with gameplay agents like AlphaGo and more recently AlphaStar. If I finish this course, will I have what I need to start work on a toy version of some gameplaying agent? If not, could you recommend where I could go next to start exploring that area?
- adamnemecek 8y ago
  The one advantage of pytorch is that you can make your graphs dynamic.
  - 0101111101 8y ago
    Tensorflow also created Eager - their dynamic environment
- joshvm 8y ago
  Once you've done fast.ai, you can look at OpenAI's crash course in reinforcement learning: https://blog.openai.com/spinning-up-in-deep-rl/ https://blog.openai.com/spinning-up-in-deep-rl/
- jph00 8y ago
  I discussed this a bit here: https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ https://www.fast.ai/2017/09/08/introducing-pytorch-for-fasta... Also some info in the fastai release post: https://www.fast.ai/2018/10/02/fastai-ai/ https://www.fast.ai/2018/10/02/fastai-ai/
- chewxy 8y ago
  If you're interested in a Go version of the AlphaGo algorithm, I wrote one: http://github.com/gorgonia/agogo http://github.com/gorgonia/agogo :)
- z0k 8y ago
  You can start with this book if you want to go straight to making a gameplay agent https://www.manning.com/books/deep-learning-and-the-game-of-go https://www.manning.com/books/deep-learning-and-the-game-of-...
whytaka 8y ago
As someone just wrapping up the FastAI Intro to Machine Learning course and moving on to DL, I'm really glad this came out now. Thanks for all the knowledge, Jeremy!
leesec 8y ago
Just wanted to say, your first course made me, someone with out a degree, extremely interested in the field, and empowered me to pursue a career in the industry. You making ML "uncool" really made me believe I could do it, when previously I had been quite self-conscious about my lack of academics. I did realize however I was still going to be quite a hard sell for companies without the credentials. But that has lead me towards obtaining a B.S. in Data Management/Analytics from WGU. Thank you for your incredible content and I look forward to completing this course. Your enthusiasm and teaching style make the class a real joy.
cs702 8y ago
In my view, this is the best, most accessible course on the planet for learning the how to of deep learning as quickly and efficiently as possible. Highly recommended!
integricho 8y ago
Could someone recommend in which order should the fast.ai courses be taken for someone new to the AI field completely? Should "Introduction to Machine Learning for Coders" be the first? or I could dive into "Practical Deep Learning" right now, followed up later perhaps by "Cutting Edge Deep Learning for Coders" ? What is the difference between the two deep learning courses also?
- nightski 8y ago
  The deep learning courses are specifically focused on deep learning whereas machine learning seems to have a broader scope. I think it just depends on what you are more interested in.
- leesec 8y ago
  If you want to get to building ML models right away, take Practical Deep Learning for Coders. Cutting Edge Deep Learning is a part 2 to that class, with more advanced topics. The Intro to Machine Learning, from my understanding, is more of a normal class that will take a deeper dive into some of the concepts and theories that aren't necessary for practical implementation, but will be helpful for understanding the whole process. I personally recommend starting with this newest course.
montenegrohugo 8y ago
This. I can tell anybody thinking about taking this course, do it! The way Jeremy Howard explains things is so down to earth and clear, you'll be surprised at how easy all this ML/DL nonsense is :) But seriously, there's a lot to learn here, and if you are looking to make your entry into the hype field of the decade I would strongly suggest starting here first (Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera course first: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning)
- cr0sh 8y ago
  I second the Andrew Ng Coursera course; I took it in 2011 when it was called "ML Class" (pre-Coursera) - yep, I was one of the beta testers!
anurag 8y ago
It's worth noting Jeremy and team are making these invaluable resources available for free, even though a lot of folks would be extremely happy to pay for them. It's amazing to me that learning materials of such exceptional quality and deep practical application are available without a paywall, especially when everyone else seems to be cashing in on the AI hype with a myriad of paid nanodegrees and such. Thank you, Jeremy and team, for everything you're doing!
baddash 8y ago
Hey, this course looks really interesting! Although I'm hesitating to take it because I'm not sure what the difference is between Machine Learning and Deep Learning. Can anyone explain?
- jhrmnn 8y ago
  DL is a branch of ML.
- patricklovesoj 8y ago
  I took half of ML before. The pace is a bit slower and topics are slightly different. ML goes into using random forests and focuses more on working on structured data. I think if you have programming experience and are more interested in Deep Learning than the Deep Learning course is great.
- alexbeloi 8y ago
  Machine Learning is a catch-all term for optimizing statistical models with data. Simplest example that you are very likely already familiar with is that of a 'best fit line' to some xy scatter plot. This starts by making an assumption (model choice) that the relationship between `x` and `y` is linear, e.g. `y=mx + b`, then you can use data (xy points) to figure out the most likely values for `m` and `b`. You can then make predictions for new `x_new` values by plugging them into your known line to get `y_new`. Machine learning often manifests in a two step process: first feature extraction, and then fitting features to a desired output. Deep learning combines these as an end-to-end process to eliminate 'human in the loop' problems that occur from feature extraction. Example: you want to predict who should win a chess game in a given board state Feature extraction (what information you think matters): what pieces does white have, what pieces does black have, is white in check, is black in check, how many valid squares can white king move to, how many valid squares can black king move, etc... * Fitting: make an assumption about the relationship between features and outcome (model choice), fit model using data (features, outcome) The Deepblue 2 model that played Kasparov used around 8000 features (not sure if this is the feature vector size or # of features). As you can imagine, feature extraction is highly dependent on expert knowledge of the problem and will often fail to cover unknown situations/cases. Deep learning models aim is to avoid limitations of expert knowledge by using raw data (e.g. occupancy of each square on a chess board) and extract features implicitly rather than relying on explicit human formulas. It has also opened up new possibilities for areas where expert knowledge has made little progress in the past (e.g. there is not much an expert can say about what pixel features are might indicate a dog/cat is contained in an image).
cr0sh 8y ago
I don't have the time to immediately work on this course, but I might later this year. What I'd like to know, as someone who has completed (in the past) the following MOOCs: 1. ML Class (plus about half of AI Class) - 2011 2. Udacity's CS373 Course - 2012 3. Udacity's Self-Driving Car Engineer Nanodegree - 2016/17 ...and gained a lot of knowledge about ML and DL thru them - would this fast.ai course be worthwhile to pursue as an addition to my prior education on the subjects? Would I find it just as challenging (from a cursory glance it looks to be a "yes", but I wanted another opinion)? I'm open to suggestions from everyone here, not just jph00/fast.ai ... Also - and this I would also like opinions on - would my time be better spent pursuing other areas I know I am not proficient in - mainly learning about probability/stats, as well as basics in calculus... For instance, in the prior courses, there was always mention of - for say backprop - stuff about the "chain rule" and other "rules" for calculus, and I feel I need to understand that more. That is, I am wanting to understand things in ML and DL at a lower level; how the "black boxes" really work. So would I be better off pursuing that, or just continuing with courses like this one? Or maybe both at the same time (if that is even possible)? I've also considered that I might need to take those subjects (probability/stats/calc) as actual courses via community college or similar (or online MOOCs). Any thoughts or suggestions are welcome!
- drageth 8y ago
  Ive watched the first two episodes of fast ai and one thing i can say is that the whole approach of this course is the opposite of what the courses you listed are structured like. Fast Ai jumps right into "how to use the script/tools" while slowly teaching the concepts behind why they work in comparison to the typical ML/AI course that involves complex learning calculus and statistics before doing anything practical.
  - deleted 8y ago
    [deleted]
- jph00 8y ago
  Whilst it's absolutely not required knowledge for most deep learning, if you are interested in more fully understanding the chain rule, my friend Terence Parr (the ANTLR guy) and I wrote a fairly thorough tutorial for matrix calculus https://explained.ai/matrix-calculus/index.html https://explained.ai/matrix-calculus/index.html .
infinitone 8y ago
Great course, looking forward to checking it out. I skimmed thru the overview and dismayed to see nothing on object detection still :/
- jph00 8y ago
  Object detection is in part 2. You can use the 2018 part 2 for now. We had hoped to move it to part 1 this year, but couldn't figure out a way to make it easy enough yet.
patricklovesoj 8y ago
This is amazing! :) I took portions of it before but motivates me to go back and work on it again. Is it just me or the overview notes from Lesson 6 & 7 are the same?
- patricklovesoj 8y ago
  Also wanted to add that as someone who hasn't professionally coded or have a degree in CS, the learning curve was a bit steep (which is why I took a break from taking the course). Are there any resources or good ways to get up to speed on getting sufficient coding experience/knowledge so that I can really digest the content?
  - jph00 8y ago
    There's a lot of Python learning resources here: https://forums.fast.ai/t/recommended-python-learning-resources/26888 https://forums.fast.ai/t/recommended-python-learning-resourc... My main recommendation is to try to do as many things as you can with code, instead of manually or with domain-specific tools. Even if it's slower at first, solving your own problems with code is the best way to become a productive coder.
ScoutOrgo 8y ago
I've gone through each iteration of the fast.ai courses and must say that it is worth it every time. The latest 2019 courses are again excellent. Even though I've covered the concepts before, I always learn a ton from every lesson. Although it is recommended to dedicate ~10 hours per lesson, it is very possible to go through each video, then circle back to focus on one of the concepts/problems presented. I recommend giving the first lesson a try. It is very rewarding feeling like you can actually do or build something at the end of each lesson.
- faitswulff 8y ago
  If you haven't done so already, would you recommend going through the old lessons? Or is the current version good enough?
  - brtknr 8y ago
    The courses move with the fastai and Pytorch libraries so its best to do the latest course so you don’t run into features that have been deprecated or renamed or miss out on new features.
hkhanna 8y ago
Hey Jeremy, just FYI this release seems to have broken the links to the Introduction to Machine Learning for Coders course. For example, I can no longer access it from the fast.ai front page (I get a 404 error.)
- jph00 8y ago
  Many thanks for letting me know - I've fixed that link now. FYI the ML course is now at: http://course18.fast.ai/ml http://course18.fast.ai/ml
  - ganeshkrishnan 8y ago
    Thank you for the link. It's currently broken here : https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ https://www.fast.ai/2017/09/08/introducing-pytorch-for-fasta... at this line: "As we developed our second course, Cutting-Edge Deep Learning for Coders, we started to hit the limits of the libraries "
byt143 8y ago
Can we have one in Julia please? https://julialang.org/blog/2019/01/fluxdiffeq https://julialang.org/blog/2019/01/fluxdiffeq
- mark_l_watson 8y ago
  +1 for the idea I use Python and TensorFlow at work but for at home side projects I am very much enjoying Julia and Flux. I haven’t tried FluxDiffEq yet. Jeremy, please :-)
mandeepj 8y ago
The lessons are fairly brief compared to Coursera\Udacity
- jph00 8y ago
  About 14 hours of lessons per part for fast.ai. A lot of Udacity courses I've seen are shorter than that. I certainly wouldn't call it "brief"!
kevintb 8y ago
fast.ai courses are EXCELLENT. I've read/watched many interviews with both creators and have been even more impressed by their drive to bring deep learning to everyone, not just the AI specialists.
mlboss 8y ago
The thing that separates fastai course from other courses is the use of fastai library (built on top of pytorch). fastai library makes it dead simple to train neural network faster and with more accuracy. Write a neural network is easy. But tweaking hyper parameters takes lot of time and knowledge. fastai library implements cyclical learning rates which varies the learning rate during training along with some other defaults for other hyperparameters. https://github.com/fastai/fastai https://github.com/fastai/fastai
jfrankamp 8y ago
Thanks for posting this, I have started working up some cloudformation scripts with the goal of getting spot instances into play for GPU nodes on AWS @ https://github.com/frankamp/fast-ai-aws-advanced https://github.com/frankamp/fast-ai-aws-advanced not quite there yet, but using layering to get separation of concerns means a user can swap out for a bigger gpu temporarily without losing anything. Cheers.
hx2a 8y ago
I finished the 2018 edition of the class two weeks ago but I am happy to go through the latest version now. The class was so thought provoking. I came up with a lot of good ideas for my thesis project while watching the videos.
carbocation 8y ago
The course is amazing. I was frustrated in late 2018 when they promoted their code to "v1" but kept iterating with breaking changes on a literal daily basis. I have stopped using their library because of this+insufficient time. However, their work is so amazing that I will pick it up again when I have more time. Yes, it's so good that I will deal with daily breaking changes to use it.
- jph00 8y ago
  The changes were just during the development phase. Should be quite settled now.
  - carbocation 8y ago
    Wonderful. I should add that you and Sebastian were always extremely responsive and helpful troubleshooting issues with that popped up due to the breaking changes.
lettergram 8y ago
Well, very similar to one I just released (even in name)[1]. Specifically, I targeted taking one problem through several network iterations. I think it can be overwhelming to go through all those different networks. Once you get the basics new networks don’t aren’t really too difficult. I haven’t taken their class before (and wouldn’t want to spend money). I think that’s a huge barrier, there’s not much of a reason to require setup on sagemaker. [1] https://austingwalters.com/neural-networks-to-production-from-an-engineer/ https://austingwalters.com/neural-networks-to-production-fro...
- jph00 8y ago
  It's free and there are no ads, and you can run it for free on colab or GCP. I have no idea about where your sagemaker comment comes from.
deleted 8y ago
[deleted]
jph00 8y ago
FYI, Google just completed adding all the pre-requisites for this course to their Colab service, which provides a GPU-powered Jupyter Notebook for free. There are no usage restrictions, other than that each session must be no longer than 12 hours (after 12 hours, you can just create a new session). So this means you can complete the whole course, and do your own projects, even if you don't have a credit card. Here's the setup guide: https://course.fast.ai/start_colab.html https://course.fast.ai/start_colab.html
- lelima 8y ago
  That's great! I almost burn up my laptop running the NLP classes and wanted something more scalable that paperspace(is still a great way to run fastai). Thanks Jeremy!
bigmit37 8y ago
Any recommendations for getting up to date with Pytorch? I could pick some cheap books from packt or maybe good blogs ?
oarabbus_ 8y ago
Does anyone else find it potentially problematic we have all these "get coding fast - for coders!" type resources available, when most programmers don't have the mathematical foundations to truly understand ML/AI/DL? It's funny, programmers scoff at non-technical folk using drag-and-drop tools to replicate certain programming functionality but I see the same thing happening here.

556 points by jph00 8y ago

DTE 8y ago

Congrats, Jeremy and crew!

jph00 8y ago
Many thanks. I'm really happy with how it turned out. I hope you all like it! :)
- input_device 8y ago
  I'm really interested in learning the math basics for deep learning. Is there an online guide that you can point us to if we want to learn the basics of calculus, linear algebra, probability theory? In other words is there a "fast.ai" version of "math for deep learning" out there?
  - joshvm 8y ago
    Yes for linear algebra, though it's probably best suited to people with at least some exposure to matrix/vector math (for example you've used SVD but have no idea how or why it works): https://www.fast.ai/2017/07/17/num-lin-alg/ https://www.fast.ai/2017/07/17/num-lin-alg/ For probability, machine learning is more about statistics (the two are related, but courses explicitly about probability will cover different things), so I would lean towards that. An Introduction to Statistical Learning in R (ISLR) is a frequently recommended book. You can ignore the R and do the exercises in Python. If you actually want to learn about probability, you can look at MIT's course: https://ocw.mit.edu/resources/res-6-012-introduction-to-probability-spring-2018/ https://ocw.mit.edu/resources/res-6-012-introduction-to-prob... EDIT: If you've never been exposed to calculus, many people swear by Khan Academy's videos.
    - input_device 8y ago
      This is great. Thanks you!
      - Canadauni 8y ago
        I would also recommend the MIT calculus courses on edX. It has been a great refresher for me since the last I looked at calc was 8 years ago.
        input_device 8y ago
        I'll check it out, thanks!

sigfubar 8y ago

Does your definition of "coders" include those of us with at most middle school mathematics knowledge? I've tried some machine learning courses before, but couldn't make sense of the vast majority of information.

jph00 8y ago
Yes it'll be a bit more work for you, but it should be OK. I provide links to more math background where required in the course.
- mywrathacademia 8y ago
  Can the exercises in the fast.ai deep learning course be done in another language other than python? And how much time in weeks should it take to complete the course? If my laptop doesn't have a GPU, can I still do all exercises?
  - rasmus1610 8y ago
    The whole course builds upon the fastai library written in python so there is no point in not using python. But I feel like you can gain a pretty good understanding of deep learning in general by watching the videos. The live course was held over a time of 7 weeks. It should take around 70h to finish it in total but I really depends on how deep you want to dive in and how easy it is for you to understand.
  - cr0sh 8y ago
    > If my laptop doesn't have a GPU, can I still do all exercises? To answer this, while technically you could do the exercises, it is possible that it will take longer (perhaps much longer) to train models and whatnot without a GPU. Where you'll really have a hard time will be anything for deep learning and large neural networks. What might take a few seconds or minutes to train on a GPU might take hours with just a CPU with a few cores. But not to worry - in the course it explains that you can use a variety of "cloud based" GPU systems that are fairly cheap to access (provided you have a credit card). So if you can't get access to a GPU, there are other ways that won't put you in the poorhouse (just make absolutely sure to completely shutdown and/or delete your VM instances you spin up). Also - while you could use another language, it is best to stick with Python for now, because it has become almost the defacto language for ML and Deep Learning, at least as far as for education and such. Mainly because it has such great libraries for scientific computing available, plus it is easily approachable. It also has a great interface to Tensorflow (and ultimately CUDA and NVidia GPUs - also, get used to the fact that if you do anything DL related, you will be using NVidia almost exclusively; while AMD and others have their own DL hardware, it isn't nearly as well supported - maybe it will be better supported in the future). That said - Tensorflow does have decent support in a number of languages, especially C++ - so if you are more familiar with that language, you'll be set to do some amazing things once you complete the course. But Python, again, is really where it's at. Tensorflow is a great library that abstracts away a lot of the pain in dealing with CUDA. But even it has pain - and there are other libraries layered over it that make things even easier to work with (ie - Keras). Seriously; in the training I've had with DL (mainly thru Udacity), we had to learn how to build a simplified DL "library" of methods for building simple neural networks. Once you learn about Tensorflow, you will have an epiphany of "OMG so much easier!". Then - when you learn about Keras, it's yet another epiphany - it makes Tensorflow look "difficult". Of course, it's best to learn in this manner if you can, so that you understand how the lower levels work - instead of everything being a black-box. Of course, at some level there will be "black-boxes"; for instance, you may or may not want to learn how Tensorflow interfaces with CUDA. Or how CUDA interfaces with the GPU. Or certain other methods in Tensorflow or some other library that perhaps wraps certain methods and functionality. Honestly, things are much nicer today than they were even 5 years ago. You can decide just what level of depth you want to explore down to.

knicholes 8y ago

So we should watch these newer ones instead of the older ones?

jph00 8y ago
For sure. It's a giant step over last year's course.
- jonahx 8y ago
  This is awesome, many thanks. Jeremy, I know you're a fellow fan of J -- is it possible to do modern ML with J? It seems like numpy is just a worse version of the array paradigm from J/APL, but I'm guessing all the specialized ML packages are the real value of python here? Any idea of anyone has done similar work in J?
- amch 8y ago
  Has the classic ML course been deprecated? I noticed that following link no longer works: https://course.fast.ai/lessonsml1/lesson1.html https://course.fast.ai/lessonsml1/lesson1.html EDIT: looks like they've been moved to the following link instead: http://course18.fast.ai/ml.html http://course18.fast.ai/ml.html
- alienreborn 8y ago
  Is there a way to create lifestyle config for Sagemaker manually instead of using fast.ai created IAM ARN and config?
- gmartres 8y ago
  What are the main differences with last year's course ? Also can we expect http://course18.fast.ai/part2.html http://course18.fast.ai/part2.html to be updated in the near future ?

2bitencryption 8y ago

I'm so excited to dive into this. One question: is there a reason you opted for PyTorch over Keras? I had the impression that Keras was the go-to for "easy as ABC" neural networking. EDIT: bonus question! I'm really fascinated with gameplay agents like AlphaGo and more recently AlphaStar. If I finish this course, will I have what I need to start work on a toy version of some gameplaying agent? If not, could you recommend where I could go next to start exploring that area?

adamnemecek 8y ago
The one advantage of pytorch is that you can make your graphs dynamic.
- 0101111101 8y ago
  Tensorflow also created Eager - their dynamic environment
joshvm 8y ago
Once you've done fast.ai, you can look at OpenAI's crash course in reinforcement learning: https://blog.openai.com/spinning-up-in-deep-rl/ https://blog.openai.com/spinning-up-in-deep-rl/
jph00 8y ago
I discussed this a bit here: https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ https://www.fast.ai/2017/09/08/introducing-pytorch-for-fasta... Also some info in the fastai release post: https://www.fast.ai/2018/10/02/fastai-ai/ https://www.fast.ai/2018/10/02/fastai-ai/
chewxy 8y ago
If you're interested in a Go version of the AlphaGo algorithm, I wrote one: http://github.com/gorgonia/agogo http://github.com/gorgonia/agogo :)
z0k 8y ago
You can start with this book if you want to go straight to making a gameplay agent https://www.manning.com/books/deep-learning-and-the-game-of-go https://www.manning.com/books/deep-learning-and-the-game-of-...

whytaka 8y ago

As someone just wrapping up the FastAI Intro to Machine Learning course and moving on to DL, I'm really glad this came out now. Thanks for all the knowledge, Jeremy!

leesec 8y ago

Just wanted to say, your first course made me, someone with out a degree, extremely interested in the field, and empowered me to pursue a career in the industry. You making ML "uncool" really made me believe I could do it, when previously I had been quite self-conscious about my lack of academics. I did realize however I was still going to be quite a hard sell for companies without the credentials. But that has lead me towards obtaining a B.S. in Data Management/Analytics from WGU. Thank you for your incredible content and I look forward to completing this course. Your enthusiasm and teaching style make the class a real joy.

cs702 8y ago

In my view, this is the best, most accessible course on the planet for learning the how to of deep learning as quickly and efficiently as possible. Highly recommended!

integricho 8y ago

Could someone recommend in which order should the fast.ai courses be taken for someone new to the AI field completely? Should "Introduction to Machine Learning for Coders" be the first? or I could dive into "Practical Deep Learning" right now, followed up later perhaps by "Cutting Edge Deep Learning for Coders" ? What is the difference between the two deep learning courses also?

nightski 8y ago
The deep learning courses are specifically focused on deep learning whereas machine learning seems to have a broader scope. I think it just depends on what you are more interested in.
leesec 8y ago
If you want to get to building ML models right away, take Practical Deep Learning for Coders. Cutting Edge Deep Learning is a part 2 to that class, with more advanced topics. The Intro to Machine Learning, from my understanding, is more of a normal class that will take a deeper dive into some of the concepts and theories that aren't necessary for practical implementation, but will be helpful for understanding the whole process. I personally recommend starting with this newest course.

montenegrohugo 8y ago

This. I can tell anybody thinking about taking this course, do it! The way Jeremy Howard explains things is so down to earth and clear, you'll be surprised at how easy all this ML/DL nonsense is :) But seriously, there's a lot to learn here, and if you are looking to make your entry into the hype field of the decade I would strongly suggest starting here first (Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera course first: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning)

cr0sh 8y ago
I second the Andrew Ng Coursera course; I took it in 2011 when it was called "ML Class" (pre-Coursera) - yep, I was one of the beta testers!

anurag 8y ago

It's worth noting Jeremy and team are making these invaluable resources available for free, even though a lot of folks would be extremely happy to pay for them. It's amazing to me that learning materials of such exceptional quality and deep practical application are available without a paywall, especially when everyone else seems to be cashing in on the AI hype with a myriad of paid nanodegrees and such. Thank you, Jeremy and team, for everything you're doing!

baddash 8y ago

Hey, this course looks really interesting! Although I'm hesitating to take it because I'm not sure what the difference is between Machine Learning and Deep Learning. Can anyone explain?

jhrmnn 8y ago
DL is a branch of ML.
patricklovesoj 8y ago
I took half of ML before. The pace is a bit slower and topics are slightly different. ML goes into using random forests and focuses more on working on structured data. I think if you have programming experience and are more interested in Deep Learning than the Deep Learning course is great.
alexbeloi 8y ago
Machine Learning is a catch-all term for optimizing statistical models with data. Simplest example that you are very likely already familiar with is that of a 'best fit line' to some xy scatter plot. This starts by making an assumption (model choice) that the relationship between `x` and `y` is linear, e.g. `y=mx + b`, then you can use data (xy points) to figure out the most likely values for `m` and `b`. You can then make predictions for new `x_new` values by plugging them into your known line to get `y_new`. Machine learning often manifests in a two step process: first feature extraction, and then fitting features to a desired output. Deep learning combines these as an end-to-end process to eliminate 'human in the loop' problems that occur from feature extraction. Example: you want to predict who should win a chess game in a given board state Feature extraction (what information you think matters): what pieces does white have, what pieces does black have, is white in check, is black in check, how many valid squares can white king move to, how many valid squares can black king move, etc... * Fitting: make an assumption about the relationship between features and outcome (model choice), fit model using data (features, outcome) The Deepblue 2 model that played Kasparov used around 8000 features (not sure if this is the feature vector size or # of features). As you can imagine, feature extraction is highly dependent on expert knowledge of the problem and will often fail to cover unknown situations/cases. Deep learning models aim is to avoid limitations of expert knowledge by using raw data (e.g. occupancy of each square on a chess board) and extract features implicitly rather than relying on explicit human formulas. It has also opened up new possibilities for areas where expert knowledge has made little progress in the past (e.g. there is not much an expert can say about what pixel features are might indicate a dog/cat is contained in an image).

cr0sh 8y ago

I don't have the time to immediately work on this course, but I might later this year. What I'd like to know, as someone who has completed (in the past) the following MOOCs: 1. ML Class (plus about half of AI Class) - 2011 2. Udacity's CS373 Course - 2012 3. Udacity's Self-Driving Car Engineer Nanodegree - 2016/17 ...and gained a lot of knowledge about ML and DL thru them - would this fast.ai course be worthwhile to pursue as an addition to my prior education on the subjects? Would I find it just as challenging (from a cursory glance it looks to be a "yes", but I wanted another opinion)? I'm open to suggestions from everyone here, not just jph00/fast.ai ... Also - and this I would also like opinions on - would my time be better spent pursuing other areas I know I am not proficient in - mainly learning about probability/stats, as well as basics in calculus... For instance, in the prior courses, there was always mention of - for say backprop - stuff about the "chain rule" and other "rules" for calculus, and I feel I need to understand that more. That is, I am wanting to understand things in ML and DL at a lower level; how the "black boxes" really work. So would I be better off pursuing that, or just continuing with courses like this one? Or maybe both at the same time (if that is even possible)? I've also considered that I might need to take those subjects (probability/stats/calc) as actual courses via community college or similar (or online MOOCs). Any thoughts or suggestions are welcome!

drageth 8y ago
Ive watched the first two episodes of fast ai and one thing i can say is that the whole approach of this course is the opposite of what the courses you listed are structured like. Fast Ai jumps right into "how to use the script/tools" while slowly teaching the concepts behind why they work in comparison to the typical ML/AI course that involves complex learning calculus and statistics before doing anything practical.
- deleted 8y ago
  [deleted]
jph00 8y ago
Whilst it's absolutely not required knowledge for most deep learning, if you are interested in more fully understanding the chain rule, my friend Terence Parr (the ANTLR guy) and I wrote a fairly thorough tutorial for matrix calculus https://explained.ai/matrix-calculus/index.html https://explained.ai/matrix-calculus/index.html .

infinitone 8y ago

Great course, looking forward to checking it out. I skimmed thru the overview and dismayed to see nothing on object detection still :/

jph00 8y ago
Object detection is in part 2. You can use the 2018 part 2 for now. We had hoped to move it to part 1 this year, but couldn't figure out a way to make it easy enough yet.

patricklovesoj 8y ago

This is amazing! :) I took portions of it before but motivates me to go back and work on it again. Is it just me or the overview notes from Lesson 6 & 7 are the same?

patricklovesoj 8y ago
Also wanted to add that as someone who hasn't professionally coded or have a degree in CS, the learning curve was a bit steep (which is why I took a break from taking the course). Are there any resources or good ways to get up to speed on getting sufficient coding experience/knowledge so that I can really digest the content?
- jph00 8y ago
  There's a lot of Python learning resources here: https://forums.fast.ai/t/recommended-python-learning-resources/26888 https://forums.fast.ai/t/recommended-python-learning-resourc... My main recommendation is to try to do as many things as you can with code, instead of manually or with domain-specific tools. Even if it's slower at first, solving your own problems with code is the best way to become a productive coder.

ScoutOrgo 8y ago

I've gone through each iteration of the fast.ai courses and must say that it is worth it every time. The latest 2019 courses are again excellent. Even though I've covered the concepts before, I always learn a ton from every lesson. Although it is recommended to dedicate ~10 hours per lesson, it is very possible to go through each video, then circle back to focus on one of the concepts/problems presented. I recommend giving the first lesson a try. It is very rewarding feeling like you can actually do or build something at the end of each lesson.

faitswulff 8y ago
If you haven't done so already, would you recommend going through the old lessons? Or is the current version good enough?
- brtknr 8y ago
  The courses move with the fastai and Pytorch libraries so its best to do the latest course so you don’t run into features that have been deprecated or renamed or miss out on new features.

hkhanna 8y ago

Hey Jeremy, just FYI this release seems to have broken the links to the Introduction to Machine Learning for Coders course. For example, I can no longer access it from the fast.ai front page (I get a 404 error.)

jph00 8y ago
Many thanks for letting me know - I've fixed that link now. FYI the ML course is now at: http://course18.fast.ai/ml http://course18.fast.ai/ml
- ganeshkrishnan 8y ago
  Thank you for the link. It's currently broken here : https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ https://www.fast.ai/2017/09/08/introducing-pytorch-for-fasta... at this line: "As we developed our second course, Cutting-Edge Deep Learning for Coders, we started to hit the limits of the libraries "

byt143 8y ago

Can we have one in Julia please? https://julialang.org/blog/2019/01/fluxdiffeq https://julialang.org/blog/2019/01/fluxdiffeq

mark_l_watson 8y ago
+1 for the idea I use Python and TensorFlow at work but for at home side projects I am very much enjoying Julia and Flux. I haven’t tried FluxDiffEq yet. Jeremy, please :-)

mandeepj 8y ago

The lessons are fairly brief compared to Coursera\Udacity

jph00 8y ago
About 14 hours of lessons per part for fast.ai. A lot of Udacity courses I've seen are shorter than that. I certainly wouldn't call it "brief"!

kevintb 8y ago

fast.ai courses are EXCELLENT. I've read/watched many interviews with both creators and have been even more impressed by their drive to bring deep learning to everyone, not just the AI specialists.

mlboss 8y ago

The thing that separates fastai course from other courses is the use of fastai library (built on top of pytorch). fastai library makes it dead simple to train neural network faster and with more accuracy. Write a neural network is easy. But tweaking hyper parameters takes lot of time and knowledge. fastai library implements cyclical learning rates which varies the learning rate during training along with some other defaults for other hyperparameters. https://github.com/fastai/fastai https://github.com/fastai/fastai

jfrankamp 8y ago

Thanks for posting this, I have started working up some cloudformation scripts with the goal of getting spot instances into play for GPU nodes on AWS @ https://github.com/frankamp/fast-ai-aws-advanced https://github.com/frankamp/fast-ai-aws-advanced not quite there yet, but using layering to get separation of concerns means a user can swap out for a bigger gpu temporarily without losing anything. Cheers.

hx2a 8y ago

I finished the 2018 edition of the class two weeks ago but I am happy to go through the latest version now. The class was so thought provoking. I came up with a lot of good ideas for my thesis project while watching the videos.

carbocation 8y ago

The course is amazing. I was frustrated in late 2018 when they promoted their code to "v1" but kept iterating with breaking changes on a literal daily basis. I have stopped using their library because of this+insufficient time. However, their work is so amazing that I will pick it up again when I have more time. Yes, it's so good that I will deal with daily breaking changes to use it.

jph00 8y ago
The changes were just during the development phase. Should be quite settled now.
- carbocation 8y ago
  Wonderful. I should add that you and Sebastian were always extremely responsive and helpful troubleshooting issues with that popped up due to the breaking changes.

lettergram 8y ago

Well, very similar to one I just released (even in name)[1]. Specifically, I targeted taking one problem through several network iterations. I think it can be overwhelming to go through all those different networks. Once you get the basics new networks don’t aren’t really too difficult. I haven’t taken their class before (and wouldn’t want to spend money). I think that’s a huge barrier, there’s not much of a reason to require setup on sagemaker. [1] https://austingwalters.com/neural-networks-to-production-from-an-engineer/ https://austingwalters.com/neural-networks-to-production-fro...

jph00 8y ago
It's free and there are no ads, and you can run it for free on colab or GCP. I have no idea about where your sagemaker comment comes from.

deleted 8y ago

[deleted]

jph00 8y ago

FYI, Google just completed adding all the pre-requisites for this course to their Colab service, which provides a GPU-powered Jupyter Notebook for free. There are no usage restrictions, other than that each session must be no longer than 12 hours (after 12 hours, you can just create a new session). So this means you can complete the whole course, and do your own projects, even if you don't have a credit card. Here's the setup guide: https://course.fast.ai/start_colab.html https://course.fast.ai/start_colab.html

lelima 8y ago
That's great! I almost burn up my laptop running the NLP classes and wanted something more scalable that paperspace(is still a great way to run fastai). Thanks Jeremy!

bigmit37 8y ago

Any recommendations for getting up to date with Pytorch? I could pick some cheap books from packt or maybe good blogs ?

oarabbus_ 8y ago

Does anyone else find it potentially problematic we have all these "get coding fast - for coders!" type resources available, when most programmers don't have the mathematical foundations to truly understand ML/AI/DL? It's funny, programmers scoff at non-technical folk using drag-and-drop tools to replicate certain programming functionality but I see the same thing happening here.