Are you extrapolating the improvements from gpt-2 to gpt-3 to gpt-4? The progress is pretty incredible. And this is without any major fundamental breakthroughs

Are you extrapolating the improvements from gpt-2 to gpt-3 to gpt-4? The progress is pretty incredible. And this is without any major fundamental breakthroughs. Definitely some great iterative improvements, but it is still all based on transformers. What happens when something significantly better than transformers is invented?