4 ms·
The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans. LLMs
by simonh 3y ago
The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans.
LLMs aren’t the be all and end all of AI though. They’re an impressive but inherently limited stepping stone, with a very constrained scope of applicability and capabilities. There is no reason to suppose that future, much more advanced architectures can’t surpass humans in reasoning ability.
Advanced AI doesn’t have to be omniscient fir us not yon be able to predict it’s capabilities, it just needs to be smarter than us. By definition if something is smarter than you, it’s not possible for you to anticipate and predict its behaviour. Therefore in principle it is not possible to predict whether that behaviour will be to your liking. That’s the problem.
- rst 3y agoMore specifically, the article assumes that the only way "AI"s will ever be produced is the way that LLMs are -- by being trained off pre-existing human generated media, and existing as static objects after that. In particular, this assumes that the system doesn't subsequently modify itself based on further interactions with the real world (e.g., doing experiments, and taking note of the results). That's assuming an awful lot, to put it mildly.
- simonh 3y agoIt's assuming something we know to be false. AlphaZero wasn't even trained against humans at all, but still beats them at Go and Chess handily.
- wrsh07 3y agoI think the paper would similarly claim something like "a chess ai trained only on human games can't play better than the humans who played those games" I think that's false - you could imagine it playing at GM level without making mistakes that real GMs make But it's missing an important fact: chess ais aren't limited to learning only from human games. They can learn from self play. You can extend this to physics (if we can simulate physics precisely enough, there's no reason ai can't make fundamental discoveries in physics)
- lkbm 3y agoIt's also not learning from individual humans. It's learning from all the humans. Different humans have different strengths, including within chess.