5 ms·
I found the Chomsky article simplistic: his arguments against the LLM lack of logic basically boil down to that LLMs are not aware of physics or logic, or socia
by cm277 4y ago
I found the Chomsky article simplistic: his arguments against the LLM lack of logic basically boil down to that LLMs are not aware of physics or logic, or social interactions. Well, guess what, they are not. Yet.
Humans are indeed 'trained' on a much larger set of data than 'just' the whole text of the internet. Indeed orders of magnitude more: think of all of the video/audio/olfactory/social cues/physics we absorb every day of our lives through all of our senses. How is that not infinitely more data than the public available text on the Internet? What is to say that a way larger LLM-like model would not be an AGI? why should human intelligence NOT be just a statistical engine?
After all, haven't we proven already that physics is more-or-less statistics at heart (quantum mechanics)? that doesn't make physics non-deterministic at human scale. So why would the result of an LLM-like model trained on reality be not-deterministic or wrong? How is a high-confidence statistical result different than reality?
In other words, nothing in Chomsky's article proves that humans are NOT statistical pattern matching engines. I am not saying that we are BTW, but it's an intriguing possibility that should be researched. After all, statistical methods and ML have made a lot more progress in the last 10-15 years than the MIT-driven, deterministic school of thought did in the previous 30. Cambridge lost, plain and simple.
- slowmovintarget 4y agoThere is no mechanism for awareness with LLMs, though. Advances on probabilistic search will have equal facility with hallucination as with the real. Regardless of whether Chomsky's article "proves" your negative, there is plenty of evidence that LLMs are not even as good as dogs and cats at forming a theory of the world around them. That we might draw a conclusion that dogs, cats, and people work differently from LLMs seems more like common sense.