3 ms·
What loss.backward() actually does
- oraziorillo 3mo agoFollowing up on my post on microcrad — many of you liked and supported the project, thank you for that — I published a long-form piece on automatic differentiation, where I explain backpropagation in simple terms so that everyone will (hopefully) be able to understand it. Questions/suggestions are welcome!
- imInGoodCompany 3mo agoAs someone who has only had a very basic understanding of how backpropagation works, I found this to be really easy to follow along with, so thanks for that. The article is also clearly entirely written, or at least drafted, by AI. The explanation is great so I'm not sure how much it matters. It does feel like the myriad of voices and writing styles on the Internet is slowly being drowned out, which makes me a bit sad.
- busfahrer 3mo agoFor anybody else trying to grok backprop, I found it very helpful to look at karpathy's microgpt, a barebones LLM in 200 lines of python. The autograd itself is just 40 lines, and reading that code and then trying to do a clean room implementation of it is a great exercise: https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95 https://gist.github.com/karpathy/8627fe009c40f57531cb1836010... https://karpathy.github.io/2026/02/12/microgpt/ https://karpathy.github.io/2026/02/12/microgpt/