4 ms·

>> Again, that's incorrect. In what other science could you produce a model that nearly 100% accurately reproduces what the system being modelled would generate

>> Again, that's incorrect. In what other science could you produce a model that nearly 100% accurately reproduces what the system being modelled would generate, and people would insist on saying that that doesn't really model the operation of that system? Inconsistent standards of evidence IMO. As far as I'm concerned the problem with claims of great accuracy (100%? Surely you jest?) is that they go above and beyond the evidence provided by modelling. For example, take Convolutional Neural Nets used to train classifiers for objects in images. It is easy to agree that such systems model the human labelling of sets of pixels in digital images, but then this observation is taken as evidence that the systems in question are doing something more than that; namely, that they are somehow modelling human perception. You can't just take any evidence you have and use it to support any hypothesis you like, and then call that "science". That's not science, it's wishful thinking. If you want machine learning to be scientific you have to be rigorous. But I'm sure if you start talking about scientific rigour in ICML and NeurIPs people will just laugh at you. "Here", they'll say, "we got systems worth millions, what's rigour got to do with anything?". And they'll be right. Scientific rigour has never made anyone any millions.

naasking 3y ago
> As far as I'm concerned the problem with claims of great accuracy (100%? Surely you jest?) The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. This was not a claim that LLMs produce make accurate factual statements with near 100% accuracy. You have to specifically instruct or lead an LLM into violating syntactic and grammatical correctness. > but then this observation is taken as evidence that the systems in question are doing something more than that; namely, that they are somehow modelling human perception I think in the case of language this is more likely than not, because language was an invention of the human mind. Therefore the structure of natural language reveals some fundamental properties of the human brain. Whether this also applies to vision is much less clear, not least because vision models are trained on 2D images and humans are embedded in a 3D space, and the contents of images are not products of the human brain the way language is. There is just no direct parity of reasoning here. Edit: in some sense, cameras were designed to work well with human perception otherwise they'd have no utility to humans, so there is something about human perception that's revealed, but what's revealed is considerably weaker and more indirect than the natural language case. You wrote a lot in your other replies and I will follow-up when I have access to more than just a phone and so can provide a more detailed argument and citations.
- YeGoblynQueenne 3y ago
  >> The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. I see this as overreach also. We have no idea what are the limits of human language ability, and cannot even say how much of human language LLMs can represent. It gets harder and harder to evaluate LLMs the larger they get because we have no better models of language, but that still doesn't tell us how good LLMs are as models of language, it just tells us earlier models are worse. If we still want to be scientific about it, at best what we could say is that a LLM (or any language model) models its training corpus, with some accuracy. That it models our ability to produce natural language? How can we know that, if we don't know what are the limits of that ability? Indeed, a major problem in machine learning research right now is the lack of good tests of language models' abilities. Sorry for writing too much in the other sub-thread. I'm looking forward to your comments.

naasking 3y ago

> As far as I'm concerned the problem with claims of great accuracy (100%? Surely you jest?) The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. This was not a claim that LLMs produce make accurate factual statements with near 100% accuracy. You have to specifically instruct or lead an LLM into violating syntactic and grammatical correctness. > but then this observation is taken as evidence that the systems in question are doing something more than that; namely, that they are somehow modelling human perception I think in the case of language this is more likely than not, because language was an invention of the human mind. Therefore the structure of natural language reveals some fundamental properties of the human brain. Whether this also applies to vision is much less clear, not least because vision models are trained on 2D images and humans are embedded in a 3D space, and the contents of images are not products of the human brain the way language is. There is just no direct parity of reasoning here. Edit: in some sense, cameras were designed to work well with human perception otherwise they'd have no utility to humans, so there is something about human perception that's revealed, but what's revealed is considerably weaker and more indirect than the natural language case. You wrote a lot in your other replies and I will follow-up when I have access to more than just a phone and so can provide a more detailed argument and citations.

YeGoblynQueenne 3y ago
>> The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. I see this as overreach also. We have no idea what are the limits of human language ability, and cannot even say how much of human language LLMs can represent. It gets harder and harder to evaluate LLMs the larger they get because we have no better models of language, but that still doesn't tell us how good LLMs are as models of language, it just tells us earlier models are worse. If we still want to be scientific about it, at best what we could say is that a LLM (or any language model) models its training corpus, with some accuracy. That it models our ability to produce natural language? How can we know that, if we don't know what are the limits of that ability? Indeed, a major problem in machine learning research right now is the lack of good tests of language models' abilities. Sorry for writing too much in the other sub-thread. I'm looking forward to your comments.