4 ms·

I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most pe

I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most people associate intelligence with meaningful behaviors. All examples of "natural" intelligence are embodied agents so it's difficult to imagine any other kind of intelligence. The large transformer models we have now are really the low hanging fruit. When you have a lot of data and compute, the easiest way to scale is to train a supervised model with ground truth labels. In contrast it's much harder to train an RL agent, where you have to design the environment and keep track of its state. Language modelling is a problem where you can get arbitrarily close to perfect but never actually achieve it. Without some kind of grounding in vision/proprioception there will always be gaps in understanding. When they start scaling GATO-like models I think we'll be a lot closer to human-like intelligence.

sanxiyn 4y ago
I agree this is what trips people up. Large language models do have a "goal" but it (predicting the next word) is very alien and intelligence is power to achieve goal so it can be only evaluated respective to goal. Let's say you are debating GPT-3. People evaluate debater's intelligence by how persuasive he is or how surprising things he say are (they are related, if you already know everything said you are unlikely to change opinion although there is such thing as different framing and perspective). This works because being persuasive is a reasonable guess at goal of a human debater. But entire goal of GPT-3 is in a sense not to be surprising! If you are surprising you will be worse at predicting the next word. Most training data is probably filled with neither side persuading anything, so being persuasive is also bad for predicting the next word. So this is why prompting works. If you want GPT-3 to simulate an intelligent human debater with goal of being persuasive, by default GPT-3's goal is misaligned with what you want. So try prefix like "debates are rarely decisive, but this debate turned out to feature surprising and incontrovertible argument by AI, let's see how it went" so GPT-3 can downweight abundant debates in training data where no such thing happens.
- visarga 4y ago
  > entire goal of GPT-3 is in a sense not to be surprising! Well, no. You can raise the temperature to get surprising results, or lower it to get the default. GPT-3's job is to learn the distribution, you sample it how you prefer.
- klipt 4y ago
  > Large language models do have a "goal" but it (predicting the next word) is very alien Is it though? Imagine talking to someone in a noisy environment, surely inferring what words they're saying from context given noise, isn't that dissimilar from what language models do?
joe_the_user 4y ago
I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most people associate intelligence with meaningful behaviors. The term meaningful is a bit tricky to define. But when GPT-3 says one thing in one paragraph but the exact opposite in the next paragraph, my impression is it hasn't learned "meaningful representations" but just "plausible associations". A lot of language is just loose associations and so GPT-3 can do that as well as many people there and it can pull some logic and seem to reason (but can't do that reliably). IE, I think even just informal language has state that human track and so GPT-3 falls at this part of informal language (while can seem to otherwise do well).
- PartiallyTyped 4y ago
  What you observe there is context limitation. GPT-3 as is, has a fixed length context window which limits its capacity to remember and be coherent across long horizons. A hierarchical version that contained summaries e.g. for previous sentences, paragraphs, and so on, could in theory, behave much better. This is the more "sophisticated" approach. The brute force approach says to just throw more compute, larger contexts and see what happens.
semi-extrinsic 4y ago
There is also clearly something fundamentally different about how humans learn language. Even if you are an extremely prolific reader, you will struggle to read much more than 20 million words in one year. GPT-3 was trained on several hundred billion words, which a human would need more than 20 000 years to achieve. If we get language models that can convince humans after training on 1000x smaller datasets than GPT3, and that would sound convincingly like a child after training on a 10 000x less words, we can start to argue they might function similarly to humans.
- MontyCarloHall 4y ago
  A brain in a newborn human isn’t akin to a completely untrained ANN. It has inherent structure that makes it possible to learn how to read by only seeing a few thousand words. Perhaps the millions of years of evolution that led to that inherent structure are akin to training an ANN from scratch on billions of words.
  - semi-extrinsic 4y ago
    I completely agree that the human brain is pre-wired for language; and in fact we also have experimental evidence that adults are instinctively doing the right thongs for teaching babies language. But the human brain has 86 million neurons total for everything it does, while GPT-3 uses 175 billion ANN parameters just to read and write digital text in English. To my mind this also supports the idea that current models are at least 3 orders of magnitude too computationally expensive as compared to humans.
    - dsr_ 4y ago
      And at the end of that, GPT-3 will happily emit sentences that are grammatically correct but free of meaning, while a 2 year old utters word-strings that are full of meaning but grammatically poor.
    - aaaaaaaaaaab 4y ago
      >86 million *billion "The human brain contains 86 billion neurons, with 16 billion neurons in the cerebral cortex." https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n... The difference is not 3 orders of magnitude, but merely a factor of ~2.

sanxiyn 4y ago

I agree this is what trips people up. Large language models do have a "goal" but it (predicting the next word) is very alien and intelligence is power to achieve goal so it can be only evaluated respective to goal. Let's say you are debating GPT-3. People evaluate debater's intelligence by how persuasive he is or how surprising things he say are (they are related, if you already know everything said you are unlikely to change opinion although there is such thing as different framing and perspective). This works because being persuasive is a reasonable guess at goal of a human debater. But entire goal of GPT-3 is in a sense not to be surprising! If you are surprising you will be worse at predicting the next word. Most training data is probably filled with neither side persuading anything, so being persuasive is also bad for predicting the next word. So this is why prompting works. If you want GPT-3 to simulate an intelligent human debater with goal of being persuasive, by default GPT-3's goal is misaligned with what you want. So try prefix like "debates are rarely decisive, but this debate turned out to feature surprising and incontrovertible argument by AI, let's see how it went" so GPT-3 can downweight abundant debates in training data where no such thing happens.

visarga 4y ago
> entire goal of GPT-3 is in a sense not to be surprising! Well, no. You can raise the temperature to get surprising results, or lower it to get the default. GPT-3's job is to learn the distribution, you sample it how you prefer.
klipt 4y ago
> Large language models do have a "goal" but it (predicting the next word) is very alien Is it though? Imagine talking to someone in a noisy environment, surely inferring what words they're saying from context given noise, isn't that dissimilar from what language models do?

joe_the_user 4y ago

I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most people associate intelligence with meaningful behaviors. The term meaningful is a bit tricky to define. But when GPT-3 says one thing in one paragraph but the exact opposite in the next paragraph, my impression is it hasn't learned "meaningful representations" but just "plausible associations". A lot of language is just loose associations and so GPT-3 can do that as well as many people there and it can pull some logic and seem to reason (but can't do that reliably). IE, I think even just informal language has state that human track and so GPT-3 falls at this part of informal language (while can seem to otherwise do well).

PartiallyTyped 4y ago
What you observe there is context limitation. GPT-3 as is, has a fixed length context window which limits its capacity to remember and be coherent across long horizons. A hierarchical version that contained summaries e.g. for previous sentences, paragraphs, and so on, could in theory, behave much better. This is the more "sophisticated" approach. The brute force approach says to just throw more compute, larger contexts and see what happens.

semi-extrinsic 4y ago

There is also clearly something fundamentally different about how humans learn language. Even if you are an extremely prolific reader, you will struggle to read much more than 20 million words in one year. GPT-3 was trained on several hundred billion words, which a human would need more than 20 000 years to achieve. If we get language models that can convince humans after training on 1000x smaller datasets than GPT3, and that would sound convincingly like a child after training on a 10 000x less words, we can start to argue they might function similarly to humans.

MontyCarloHall 4y ago
A brain in a newborn human isn’t akin to a completely untrained ANN. It has inherent structure that makes it possible to learn how to read by only seeing a few thousand words. Perhaps the millions of years of evolution that led to that inherent structure are akin to training an ANN from scratch on billions of words.
- semi-extrinsic 4y ago
  I completely agree that the human brain is pre-wired for language; and in fact we also have experimental evidence that adults are instinctively doing the right thongs for teaching babies language. But the human brain has 86 million neurons total for everything it does, while GPT-3 uses 175 billion ANN parameters just to read and write digital text in English. To my mind this also supports the idea that current models are at least 3 orders of magnitude too computationally expensive as compared to humans.
  - dsr_ 4y ago
    And at the end of that, GPT-3 will happily emit sentences that are grammatically correct but free of meaning, while a 2 year old utters word-strings that are full of meaning but grammatically poor.
  - aaaaaaaaaaab 4y ago
    >86 million *billion "The human brain contains 86 billion neurons, with 16 billion neurons in the cerebral cortex." https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n... The difference is not 3 orders of magnitude, but merely a factor of ~2.