People are saying no, but a lite version of this is already done. It is called foundational models. A foundational model such as Llama2 is then fine tuned at a

People are saying no, but a lite version of this is already done. It is called foundational models. A foundational model such as Llama2 is then fine tuned at a much lower cost by various people either for research or production. Caches and checkpoints will always be useful. But you are not saving every computation.