>> The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically

>> The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. I see this as overreach also. We have no idea what are the limits of human language ability, and cannot even say how much of human language LLMs can represent. It gets harder and harder to evaluate LLMs the larger they get because we have no better models of language, but that still doesn't tell us how good LLMs are as models of language, it just tells us earlier models are worse. If we still want to be scientific about it, at best what we could say is that a LLM (or any language model) models its training corpus, with some accuracy. That it models our ability to produce natural language? How can we know that, if we don't know what are the limits of that ability? Indeed, a major problem in machine learning research right now is the lack of good tests of language models' abilities. Sorry for writing too much in the other sub-thread. I'm looking forward to your comments.