5 ms·

The lessons look interesting from a high level perspective. And I think could help people guide their applications. I think there's also a need for a very low

The lessons look interesting from a high level perspective. And I think could help people guide their applications. I think there's also a need for a very low level course in deep learning. I.e. on the level of someone who wishes to write their own deep learning library. Because from high up, sure it all looks like the chain rule, but down low, it gets messy quickly if you want to write a high performance library on your own.

mlevental 7y ago
you're talking about thousands on PhD level work in autodiff and optimization and whatever else. you're not going to learn that in one PhD let alone one class.
- bigred100 7y ago
  I think that’s only necessary for a good library. I think you could probably make a bad or simple one with much less.
Datenstrom 7y ago
If you want to take a look at a framework that isn't a terrible mess and that one person can understand both Darknet [1] and Flux [2] are really nice. Learned a lot from reading their source. [1]: https://github.com/pjreddie/darknet/ https://github.com/pjreddie/darknet/ [2]: https://github.com/FluxML/Flux.jl https://github.com/FluxML/Flux.jl
- monadmancer 7y ago
  These are refreshing. Redmon's resume is, the first of its kind I suspect.
yeldarb 7y ago
Part 2 of this course, which will be released later this month, is a step by step walkthrough of building the FastAI and PyTorch libraries from scratch (minus autodiff).
umanwizard 7y ago
Not a complete answer to what you're looking for, but you might find this dive into PyTorch internals interesting: http://blog.ezyang.com/2019/05/pytorch-internals/ http://blog.ezyang.com/2019/05/pytorch-internals/
- monadmancer 7y ago
  This is actually very close to the flavor I had in mind.
baalimago 7y ago
Now, I don't know if you're implying that fast.ai specifically should include this in their course, and you're saying this as feedback, but there's tons of resources online for it. If you're looking for a guide i highly recommend this: http://neuralnetworksanddeeplearning.com/chap1.html http://neuralnetworksanddeeplearning.com/chap1.html.
siekmanj 7y ago
I've been writing a C-based deep learning library for the past year or so, which I've recently been trying to clean up to be a bit more presentable. It was pretty heavily inspired by Darknet, although it admittedly has far fewer features: https://github.com/siekmanj/sieknet https://github.com/siekmanj/sieknet
dragandj 7y ago
I recently published a series of 16 blog posts Deep Learning from Scratch to GPU that does exactly that! [0] I also started working on a book titled "Deep Learning for Programmers"[1] that takes it even further! You can subscribe today and start reading the drafts as they are written. [0] https://dragan.rocks https://dragan.rocks [1] https://aiprobook.com https://aiprobook.com

mlevental 7y ago

you're talking about thousands on PhD level work in autodiff and optimization and whatever else. you're not going to learn that in one PhD let alone one class.

bigred100 7y ago
I think that’s only necessary for a good library. I think you could probably make a bad or simple one with much less.

Datenstrom 7y ago

If you want to take a look at a framework that isn't a terrible mess and that one person can understand both Darknet [1] and Flux [2] are really nice. Learned a lot from reading their source. [1]: https://github.com/pjreddie/darknet/ https://github.com/pjreddie/darknet/ [2]: https://github.com/FluxML/Flux.jl https://github.com/FluxML/Flux.jl

monadmancer 7y ago
These are refreshing. Redmon's resume is, the first of its kind I suspect.

yeldarb 7y ago

Part 2 of this course, which will be released later this month, is a step by step walkthrough of building the FastAI and PyTorch libraries from scratch (minus autodiff).

umanwizard 7y ago

Not a complete answer to what you're looking for, but you might find this dive into PyTorch internals interesting: http://blog.ezyang.com/2019/05/pytorch-internals/ http://blog.ezyang.com/2019/05/pytorch-internals/

monadmancer 7y ago
This is actually very close to the flavor I had in mind.

baalimago 7y ago

Now, I don't know if you're implying that fast.ai specifically should include this in their course, and you're saying this as feedback, but there's tons of resources online for it. If you're looking for a guide i highly recommend this: http://neuralnetworksanddeeplearning.com/chap1.html http://neuralnetworksanddeeplearning.com/chap1.html.

siekmanj 7y ago

I've been writing a C-based deep learning library for the past year or so, which I've recently been trying to clean up to be a bit more presentable. It was pretty heavily inspired by Darknet, although it admittedly has far fewer features: https://github.com/siekmanj/sieknet https://github.com/siekmanj/sieknet

dragandj 7y ago

I recently published a series of 16 blog posts Deep Learning from Scratch to GPU that does exactly that! [0] I also started working on a book titled "Deep Learning for Programmers"[1] that takes it even further! You can subscribe today and start reading the drafts as they are written. [0] https://dragan.rocks https://dragan.rocks [1] https://aiprobook.com https://aiprobook.com