I disagree, the burden is almost exclusively maintaining fast implementations of primitive operators for all hardware. These ML libraries are collections of pu

I disagree, the burden is almost exclusively maintaining fast implementations of primitive operators for all hardware. These ML libraries are collections of pure functions with minimal interfaces. There's very little code interdependence and it's not particularly difficult to implement modern algorithms to train networks. full stable diffusion in <800 lines: https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc37786ffdafc5dc0e990df593/examples/stable_diffusion.py#L532 https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc3778... autograd in <30 lines: https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc37786ffdafc5dc0e990df593/tinygrad/tensor.py#L123-L152 https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc3778... Adam in <20 lines: https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc37786ffdafc5dc0e990df593/tinygrad/nn/optim.py#L51 https://github.com/geohot/tinygrad/blob/4fb97b8de0e210cc3778...