I think incremental training should be possible. When next model is much smaller and as an input takes latent state from the bigger model. Then training will be

I think incremental training should be possible. When next model is much smaller and as an input takes latent state from the bigger model. Then training will be faster, those smaller models can be chained. Another option is to have small re-trainable areas within the big model. Of course ideally model should be able to work with some sort of "knowledge base", which can be updated daily. As a main, or additional data source. As for me I'm looking for ways to increase productivity. Have ChatGPT Plus, and in waiting lists for GPT-4 and Bing. Neither solves big problems, but usually there are many small things which can be done faster. I would rather offload them.