"Cannot be inferred from its training set" is a pretty difficult hurdle. Human beings can infer patterns that aren't there, and we typically call those hallucin

"Cannot be inferred from its training set" is a pretty difficult hurdle. Human beings can infer patterns that aren't there, and we typically call those hallucinations or even psychoses. On the other hand, some unconfirmed, novel patterns that humans infer actually represent groundbreaking discoveries, like for example much of the work of Ramanujan. In a real sense, all of the future discoveries of mathematics already exist in the "training set" of our present understanding, we just haven't thought it all the way through yet. If we discover something new, can we say that the concept didn't exist, or that it "couldn't be inferred" from previous work? I think the same would apply to LLMs and their understanding of the way we encode information using language. Given their radically different approach to understanding the same medium, they are well poised to both confirm many things we understand intuitively as well as expose the shortcomings of our human-centric model of understanding.