4 ms·
The full quote is: In the past, I have advocated learning Deep Learning using only a matrix library. For the purposes of actually knowing what goes on under th
by joshvm 10y ago
The full quote is:
In the past, I have advocated learning Deep Learning using only a matrix library. For the purposes of actually knowing what goes on under the hood, I think that this is essential, and the lessons learned from building things from scratch are real gamechangers when it comes to the messiness of tackling real world problems with these tools. However, when building neural networks in the wild (Kaggle Competitions, Production Systems, and Research Experiments), it's best to use a framework.
Which makes a bit more sense in context. Certainly I think taking CS231N has put me at an advantage going through fast.ai because the quality of the notes (and discussion on backprop, how neural nets work, etc.) is significantly better. Andrej Karpathy and Justin Johnson have done an amazing job with the whole website.
The downside with CS231N is the lack of really real-world practice if you're want to work with image data. The implementation examples only focus on CIFAR-10. I think the two courses are definitely complementary and I think if I'd only taken fast.ai I would be left a bit unsatisfied about knowing what I'd been doing when twiddling parameters.
- nl 10y agoI'd never tell anyone not to do CS231N! However, my comment was about how much the theory helps in the real world (and I don't think my quote was misleading. I think the author still advocates learning theory before using the frameworks) I think that the fast.ai approach of showing solutions, then explaining the theory is really good. For example in lesson 2 Fast.ai shows how to finetune a VGG model to get (real!) state of the art performance on a real-wordl, two-class image recognition problem by adding an additional dense layer[1]. In CS231N (which I haven't done, although I have glanced through the notes) this isn't really said anywhere. It's kind of implied in a lot of places, and if you study it you'll be able to work it out for yourself. That isn't bad, but I'm unconvinced that gives the huge "real world" advantage given the amount of time it takes to get to that point. I strongly agree they are complimentary though. [1] http://wiki.fast.ai/index.php/Lesson_2_Notes#Adding_a_Dense_Layer http://wiki.fast.ai/index.php/Lesson_2_Notes#Adding_a_Dense_...
- joshvm 10y agoSure:) Perhaps I should have made clear that I was referring to the fact that CS231N follows the recommendation in the blog post - i.e. lots of exercises where you implement things in primitive Numpy - rather than a broad "it's a better course". CS231N is frustrating in the real-world respect, although you're given some general recipes e.g. use ReLU, use Adam, etc. That said, I'd be interested to see some actual results from people taking what they learned in fast.ai (with no prior knowledge)and applying it to a new test set. It's easy to be impressed with the examples where you're walked through it, but figuring out what works on your data is whole new kettle of fish!