Something from previous discussions about the whole "Rather than paying $50k up front for 8 x A100's, you can just rent some GPU's for $1.2k to train the whole

Something from previous discussions about the whole "Rather than paying $50k up front for 8 x A100's, you can just rent some GPU's for $1.2k to train the whole thing in 4 days" That feel off to me is that it completely ignores the compute time spent exploring new ideas, failing, tweaking the training data, etc.