Thanks for this inspiring essay, I couldn’t agree more that “reinventing for insight” is one of the best ways to learn. I had a similar experience couple months

Thanks for this inspiring essay, I couldn’t agree more that “reinventing for insight” is one of the best ways to learn. I had a similar experience couple months ago when I built an entire PyTorch-style machine learning library [1] from scratch, using nothing but Python and NumPy. I started with a tiny autograd engine, then gradually created layer modules, optimizers, data loaders etc... I simply wanted to learn machine learning from first principles. Along the way I attempted to reproduce classical convnets [2] all the way to a toy GPT-2 [3] using the library I built. It definitely helped me understand how machine learning worked underneath the hood without all the fancy abstractions that PyTorch/TensorFlow provides. Kinda like reinventing the car using the wheel I reinvented :) [1] https://github.com/workofart/ml-by-hand https://github.com/workofart/ml-by-hand [2] https://github.com/workofart/ml-by-hand/blob/main/examples/cnn.py https://github.com/workofart/ml-by-hand/blob/main/examples/c... [3] https://github.com/workofart/ml-by-hand/blob/main/examples/gpt2.py https://github.com/workofart/ml-by-hand/blob/main/examples/g...