Yes, indeedy. Many things learned over the last three decades with smaller (the current terminology is "extremely tiny"! :) neural networks are being revisited

Yes, indeedy. Many things learned over the last three decades with smaller (the current terminology is "extremely tiny"! :) neural networks are being revisited for these large models.