3 ms·
Agreed. Implementing backprop myself—even if it was a crappy, slow version in MatLab—is what finally got me to understand it. I've worked as an ML researcher fo
by mkaic 3y ago
Agreed. Implementing backprop myself—even if it was a crappy, slow version in MatLab—is what finally got me to understand it. I've worked as an ML researcher for 2 years since then and I'm still routinely happy that I have that deeper understanding of what's going on under the hood of the models I'm training.
- mlajtos 3y agoI agree that implementing autodiff is nice exercise to grasp the whole stack. But its a ladder that has to be thrown away after you climbed it. In other words, write that autodiff for yourself, and then use production grade ML solution where you won't ever write backward pass by hand. The APL paper doesn't have implementation of autodiff, rather 90s style hand derivations.