In fairness, training machine learning models is definitely one of areas where only running when it’s cheap to run makes some amount of sense. It’s something yo

In fairness, training machine learning models is definitely one of areas where only running when it’s cheap to run makes some amount of sense. It’s something you want done eventually, not something that is responding to an immediate need. In practice…yeah. The machine’s your running your models on are expensive, the people working on the model waiting extra time to check the new model are expensive, and not keeping up with your competitors because you only work half the time is not generally a wise choice for someone like Microsoft. I have worked for a smaller company that trained a model on AWS when spot instances were cheap…until they got large enough to purchase some hardware best suited to their use case that could run almost constantly, then doing everything except training their model in the cloud. So yeah I also doubt this makes sense in practice.