ai. Our models train in a reasonable time (or we push them onto gpu) but doing the pipelines and etl when developing is a big time sink. We could sit there and

ai. Our models train in a reasonable time (or we push them onto gpu) but doing the pipelines and etl when developing is a big time sink. We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers. For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere. Now htop glows like a Christmas tree when it’s running. This was costly to do but it was the only way. We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc And finally, at some point a client will just come along with another million rows of data