4 ms·

Language is a communication method evolved by intelligent beings, not a (primary) constituent of intelligence. From neurology it's pretty clear that the basic

Language is a communication method evolved by intelligent beings, not a (primary) constituent of intelligence. From neurology it's pretty clear that the basic architecture of human minds is functional interconnected neural networks and not symbolic processing. My belief is that world-modeling and prediction is the vast majority of what intelligence is, which is quite close to what the LLMs are doing. World models can be in many representations (symbolic, logic gates, neural networks) but what matters is how accurate they are with respect to reality, and how well the model state is mapped from sensory input and back into real-world outputs. Symbolic human language relies on each person's internal world model and is learned by interacting with other humans who share a common language and similar enough world models, not the other way around (learning the world model as an aspect of the language itself). Children learn which language inputs and outputs are beneficial and enjoyable to them using their native intelligence and can strengthen their world model with questions and answers that inform their model without having to directly experience what they are asking about. People who don't believe the LLMs have a world model are wrong because they are mistaking a physically weak world model for no world model. GPT-3 doesn't understand physics well enough to embed models of the referents of language into a unified model that has accurate gravity and motion dynamics, so it maintains a much more dreamlike model where objects exist in scenes and have relationships to each other but those relationships are governed by literary relationships instead of physical ones and so contradictions and superpositions and causality violations are allowed in the model. As multimodal transformers like Gato get trained on more combined language and sensory input their world models will become much more physically and causally accurate which will be reflected in their accuracy on NLP tasks.

ripe 4y ago
You might be right that these "multimodal" transformers, by integrating additional data from non-text sources, would be more capable than GPT-3. But I don't think that invalidates Gary Marcus's point. The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.
- benlivengood 4y ago
  To hint at what I fundamentally mean by model I'd be interested to see a symbolic model of vision. E.g. take 5125123 numbers and give them names and then follow some rules to arrive at "cat" or "dog". Image recognition, I think, is demonstrably not symbolic. Likewise most transformations from the real world to model state are not symbolic. Within model state, symbolism may have uses but Church-Turing claims that it isn't necessary. It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine. I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitative-reasoning.html?m=1 https://ai.googleblog.com/2022/06/minerva-solving-quantitati... I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.
noobermin 4y ago
Reading things like this on hacker news and twitter reminds me a lot like particle physicists unwilling to grapple with lack of evidence for SUSY. Every time someone points it out, these guys merely recite the holy texts (describe the model in gross detail with more magic words as if the criticism is because the critic don't understand it well enough), as if that addresses the critique at all. They then eventually gesture at the energy horizon, suggesting that "with more s[0] you'll eventually see my particles." It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either. Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally? Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach? [0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)
- Der_Einzige 4y ago
  "Every time I fire a linguist, the performance of my speech recognizer goes up"
- xwolfi 4y ago
  You say far more than any artist but I m not so sure. When we spend 20 years looking at stuff before doing one passably interesting work of art (bar absolute outlying geniuses), it's not a small time nor a small training cost. It may feel easy for us but our brain never stop ingesting information for decades before being original enough.
fspeech 4y ago
Well put on what language is. Language encodes the delta of worlds. There are way more unsaid than what is said explicitly.
hinkley 4y ago
Are you saying that GPT-3 doesn't grasp Object Permanence?

ripe 4y ago

You might be right that these "multimodal" transformers, by integrating additional data from non-text sources, would be more capable than GPT-3. But I don't think that invalidates Gary Marcus's point. The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.

benlivengood 4y ago
To hint at what I fundamentally mean by model I'd be interested to see a symbolic model of vision. E.g. take 5125123 numbers and give them names and then follow some rules to arrive at "cat" or "dog". Image recognition, I think, is demonstrably not symbolic. Likewise most transformations from the real world to model state are not symbolic. Within model state, symbolism may have uses but Church-Turing claims that it isn't necessary. It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine. I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitative-reasoning.html?m=1 https://ai.googleblog.com/2022/06/minerva-solving-quantitati... I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.

noobermin 4y ago

Reading things like this on hacker news and twitter reminds me a lot like particle physicists unwilling to grapple with lack of evidence for SUSY. Every time someone points it out, these guys merely recite the holy texts (describe the model in gross detail with more magic words as if the criticism is because the critic don't understand it well enough), as if that addresses the critique at all. They then eventually gesture at the energy horizon, suggesting that "with more s[0] you'll eventually see my particles." It's feels a bit similar here. There are clear limitations to AI by just NNs, namely they like Stable Diffusion must be trained on so much data, far more than any single artist will ever see, and they still fall short sometimes. With NLP it seems a little more obvious because the limitations seem to lag actual speech understood by humans still, even with the massive dataset again beyond what any single human will ever digest. There are clear issues here, merely reciting to critics what your model of intelligence is doesn't address the shortcomings these systems have which any literate person can perceive. And gesturing at the data horizon (you need even more data??) is not convincing either. Apparently this cat (the interviewed author) has some baggage, but the things he's talking about (symbols) have already been studied and used by philosophers and linguists long before AI or computers were a thing. It helps to build upon what other researchers have already done, I don't know why NN researchers don't seem to map what they are doing to existing research on intelligence in other areas, namely cognitive science. Do NN researchers have any contact with linguists or cognitive scientists generally? Again, being a little ignorant of his baggage, I do think a hybrid approach sounds like it would be promising. Symbolic AI failed in the 70s, NN is getting a bit of the way there now and is pretty useful but misses a few things, why go all in on just one type of approach? [0] s is basically the center of mass energy, what particle physicists mean when they say the "energy" of an interaction (collision)

Der_Einzige 4y ago
"Every time I fire a linguist, the performance of my speech recognizer goes up"
xwolfi 4y ago
You say far more than any artist but I m not so sure. When we spend 20 years looking at stuff before doing one passably interesting work of art (bar absolute outlying geniuses), it's not a small time nor a small training cost. It may feel easy for us but our brain never stop ingesting information for decades before being original enough.

fspeech 4y ago

Well put on what language is. Language encodes the delta of worlds. There are way more unsaid than what is said explicitly.

hinkley 4y ago

Are you saying that GPT-3 doesn't grasp Object Permanence?