3 ms·

I really like Sebastian's content but I do agree with you. I didn't get into deep learning until starting with Karpathy's series, which starts by creating an au

I really like Sebastian's content but I do agree with you. I didn't get into deep learning until starting with Karpathy's series, which starts by creating an autograd engine from scratch. Before that I tried learning with fast.ai, which dives immediately into building networks with Pytorch, but I noped out of there quickly. It felt about as fun as learning Java in high school. I need to understand what I'm working with!

krmboya 2y ago
Maybe it's just different learning styles. Some people, me included, like to start getting some immediate real world results to keep it relevant and form some kind of intuition, then start peeling back the layers to understand the underlying principles. With fastAI you are already doing this by the 3rd lecture. Like driving a car, you don't need to understand what's under the hood you start driving, but eventually understanding it makes you a better driver.
- BaculumMeumEst 2y ago
  For sure! In both cases I imagine it is a conscious choice where the teachers thought about the trade-offs of each option. Both have their merits. Whenever you write learning material you have to decide where to draw the line of how far you want to break down the subject matter. You have to think quite hard about exactly who you are writing for. It's really hard to do!
  - jph00 2y ago
    You seem to be implying that the top-down approach is a trade off that involves not breaking down the subject matter into as lower level details. I think the opposite is true - when you go top down you can keep teaching lower and lower layers all the way down to physics if you like!
jph00 2y ago
fast.ai also does autograd from scratch - and goes further than Karpathy since it even does matrix multiplication from scratch. But it doesn’t start there. It uses top-down pedagogy, instead of bottom up.
- BaculumMeumEst 2y ago
  Oh that’s interesting to know! I guess I gel better with bottom up. As soon as I start seeing API functions I don’t understand I immediately want to know how they work!

krmboya 2y ago

Maybe it's just different learning styles. Some people, me included, like to start getting some immediate real world results to keep it relevant and form some kind of intuition, then start peeling back the layers to understand the underlying principles. With fastAI you are already doing this by the 3rd lecture. Like driving a car, you don't need to understand what's under the hood you start driving, but eventually understanding it makes you a better driver.

BaculumMeumEst 2y ago
For sure! In both cases I imagine it is a conscious choice where the teachers thought about the trade-offs of each option. Both have their merits. Whenever you write learning material you have to decide where to draw the line of how far you want to break down the subject matter. You have to think quite hard about exactly who you are writing for. It's really hard to do!
- jph00 2y ago
  You seem to be implying that the top-down approach is a trade off that involves not breaking down the subject matter into as lower level details. I think the opposite is true - when you go top down you can keep teaching lower and lower layers all the way down to physics if you like!

jph00 2y ago

fast.ai also does autograd from scratch - and goes further than Karpathy since it even does matrix multiplication from scratch. But it doesn’t start there. It uses top-down pedagogy, instead of bottom up.

BaculumMeumEst 2y ago
Oh that’s interesting to know! I guess I gel better with bottom up. As soon as I start seeing API functions I don’t understand I immediately want to know how they work!