I wonder how much of the formidable amount of computing resources required for deep learning can be attributed to wasteful and inefficient programming practices

I wonder how much of the formidable amount of computing resources required for deep learning can be attributed to wasteful and inefficient programming practices. A lot of the ML libraries that I see are written in Python with very little attention paid to aspects such as memory usage, cache coherency, concurrency, etc. If we focused on writing more efficient software instead of demanding bigger and faster machines with more and more GPUs, would the cost of ML become more practical? More importantly, as the author pointed out, would smaller companies have a better chance at making advancements in the field?