Problem is that most technologies don't hit a visible "dead end". Look at NLP before transformers, cars, planes, steel tech, wood tech and even books. What you

Problem is that most technologies don't hit a visible "dead end". Look at NLP before transformers, cars, planes, steel tech, wood tech and even books. What you have is a steady slowdown in the number of revolutionary discoveries and just long list of marketing hyped small improvements. There are fundamental limitations with transformers that will not go away for as long as AI equates transformers. The first one is the lack of understanding/control by humans. Orgs want guarantees that these systems won't behave unexpectedly while also wanting innovative and useful behaviour from them. Ultimately, most if not all neural nets are black boxes so understanding the reasoning for a specific action at a given time, let alone their behaviour in general is just not feasible due to their sheer complexity. We just don't understand why the behave the way they do in a scientific way anyway than we understand why a specific rabbit did a specific action at that particular moment in a way that can be used to make accurate predictions about when it will do that action again. Due to our lack of understanding, we just cannot control these things accurately. We either block some of their useful abilities to reduces the changes of undesired behaviour or you are you exposed to it. This trade-off is just a fundamental limitation of the fact that transformers are used nowadays are neural nets and as such have all the limitations that they have. The second one is our inability to completely stop the hallucinations. From what my understanding, this is inherently tied to the very nature of how transformers based LLMs produce output. There is no understanding of the notion of truth or real world. It's just emulating patterns seeing in its training data, it just so happens that some of those don't correlate with real world facts (truth) even if they correlate with human grammar. In so far as there is no understanding of the notion of truth as separate from patterns in data, however implicit, hallucinations will continue. And there is no reason to believe that we will come up with a revolutionary way to train these systems in a way that they understand truth and not just grammar. The third one is learning, models can't learn or remember as such, context learning is a trick to emulate learning but it's extremely inefficient and not scalable and models don't really have the ability to manipulate it the way humans or other animals can do. This is probably the most damning of them all as you cannot possible have a human level General Artificial that is unable to learn new skills on its own. I would bet money on there not being significant progress before 2030. By significant progress I mean, the ability to do something that they could not do before at all regardless of the amount of training thrown at them given the same computing resources we have now.