It seems to me we'll see large glue model, which will be able to read from model last layer and write directly into model first layer and are trained at composi

It seems to me we'll see large glue model, which will be able to read from model last layer and write directly into model first layer and are trained at composing them. Some is already happening, and it cannot be generalized, but the idea of being able to glue already trained medium sized model together and then do a retrain of the whole thing to improve performances seems already establishing itself in products and will allow much efficient expenditure of training resources.