1) The AI's aren't getting much better since when? Last week? You appear to be assuming that the only way to make current LLM-based AI's better is by feeding t

1) The AI's aren't getting much better since when? Last week? You appear to be assuming that the only way to make current LLM-based AI's better is by feeding them more and more data, but that's simply not true. These current (really first generation, not withstanding GPT's versioning) AI's are very data/parameter inefficient, and do not differentiate between data used to develop reasoning and data that is just pure knowledge. The short-term path forward is more judicious training set design to use less data, and require less model parameters, so that the model itself just learns to reason, and doesn't need to do double-duty as a database repository of facts as well. Retrieval-based plugins and similar approaches can be used to give access to external knowledge. 2) There will be many different sizes and capabilities of AI. They don't all need to be superhuman. We are entering the AI age and just as the computer age didn't result in everyone needing a mainframe or datacenter in their garage (although there are uses for them), the AI age will not result in everyone running the world's most powerful super-human AI on their smartphone. There will be a range of AI's for different uses cases. Current models are also very inefficient. We are essentially day 1 into the NN/LLM AI era. Think of ChatGPT as the ENIAC of the age. DeepMind's Chinchilla scaling laws have already proven how inefficient these systems are, with a suitably trained 70B model equaling the performance of a 175B one. There is tons more work to be done on efficiency and smarter model design. 3). Perhaps, but per point 1) we're not running out of data anyway, and there will anyways always be people/companies happy to sell data for a price, even if it is not for free. Future AI's will learn the same way we do by experimentation and curiosity/surprise, and will not need to be spoon fed a training set in the same way as today's systems. 4) No idea what you're talking about here. NN optimization is hardly spent as a technique - that'd be like saying "we've reached the limit of what we can to with software development - we need a new idea". LLMs as a path to AI, or building block of AI, are of course a very new idea - a few years old at best. A few decades ago no-one was even working on neural nets (current DeepLearning revolution basically started with AlexNet in 2012) .. it was all GOFAI symbolic systems - failed approaches like SOAR and CYC. If you compare the Transformer architecture to the human cortex, and our brain's overall cognitive architecture, some of the missing pieces are extremely obvious. We're not at any sort of impasse as to how to move things forward.