Right, I think it's a limitation of deep learning. Transformers and gigantic leaps in scaling have allowed AI models to reach impressive capabilities, but at it

Right, I think it's a limitation of deep learning. Transformers and gigantic leaps in scaling have allowed AI models to reach impressive capabilities, but at its core it's limited by its training data. This is not how actual intelligence works. People don't need terabytes of information to learn how to communicate and reason. For this reason the current AI trend will have a bigger impact on creative tasks, rather than critical technical ones. It's already great for generating art quickly, at least for the concepting phase, and other creative assets that can afford to be generic. Solving technical problems, on the other hand, requires reasoning beyond what can be extracted from training data. We'll need a new paradigm of AI in order to have a chance at creating models that properly reason. Even without detail knowledge of the brain, we can safely speculate that the reason and language areas are extremely efficient compare to cutting edge LLMs, which means there are algorithms more complex and efficient than simple artificial neural connections that just sum weights with a bias.