GPT et al are piggybacking on human knowledge in a sense, by using the "predict the next word" method of encapsulating intelligence. But this method of learning

GPT et al are piggybacking on human knowledge in a sense, by using the "predict the next word" method of encapsulating intelligence. But this method of learning imposes a ceiling on the level they can reach - that of the material they are able to train with. They can achieve extraordinary breadth of knowledge from the vast corpus of human written text, but in order to exceed human intelligence in "height", they will need to work directly with nature, as humans do. This, it seems to me, will be where some new ideas will be needed.