The constant improvements of SOTA are the main thing keeping the investment machine running. We can't really remove training costs from inference costs, because

The constant improvements of SOTA are the main thing keeping the investment machine running. We can't really remove training costs from inference costs, because a bunch of the funding and loans for the inference hardware only exists because the promises the continuous training (tries to) provides.