7 ms·

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qu

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

chriswarbo 4y ago
> I have yet to see an output from a big language model that doesn’t just look like P(text|internet) True, but the same can be said of many things; e.g. biology just looks like P(reproduction|environment), the economy just looks like P(profit|markets), etc. There can still be rich structure inside, and useful abstractions to describe them.
- benreesman 4y ago
  Yeah and I hope I didn’t come off like I was trying to knock the technical achievement: it’s remarkable along multiple dissensions: at a minimum technical, infrastructural, mathematical (you don’t throw 25-50k A100s for months at something without running some serious numbers first). It’s possible that I’ve just fallen too far under the influence of Deutsch and Marletto, but as someone who has worked on systems like this I’m rather skeptical that one of these things is going to break the gridlock between quantum theory and general relatively any time soon. There’s no reason why one couldn’t principle but I’ve babysat enough big ML systems that I tend to think in terms of “how do we keep this thing from shitting itself” rather than “damn this thing is going to win a Fields medal if I turn my back on it”.
  - sinenomine 4y ago
    It is common to switch to a conservative mode of thought upon entering one's domain of competence; The question is, though, if the expertise and intuition developed during, say, running XGboost classifiers at scale in AdTech really of much relevance when thinking about large transformer models trained with a self-supervised objective and RLHF? If you try to study this in depth, these models can do something the usual "datascience"-tier ones commonly cannot: https://arxiv.org/abs/2205.10343 https://arxiv.org/abs/2205.10343 https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ https://moultano.wordpress.com/2020/10/18/why-deep-learning-...
    - benreesman 4y ago
      If you’re curious about how we used ensembled weak learners to form a strong enough learner to kick the shit out of everyone except DoubleClick in AdTech, this is a decent start: https://en.m.wikipedia.org/wiki/Ensemble_learning https://en.m.wikipedia.org/wiki/Ensemble_learning. By the standards of AdTech, latent space stuff was pretty unproven when I left the game a few years ago. Low-rank approximations were in some sense implicit in that, but were not at that time an explicit goal. If I were doing an AdTech system from scratch I’d most likely reach for the kind of recommender systems that you’re alluding to sooner. Sounds like you know your stuff :)
- sandgiant 4y ago
  I don't think anyone looks a biological systems and thinks they are well described by P(reproduction|environment). ChatGPT, on the other hand, is P(text|internet) more or less by definition (it's more than that, but not much more). The fact that we observe what looks like emergent structures such as "understanding", "knowledge" or "reasoning" is fantastic, but it is not in any way incompatible with a P(text|internet) model simply "mimicking" humans. I agree that exploring the inner workings of these emergent features is interesting in it's own right, but all that glitters is not gold.
ImprobableTruth 4y ago
The article links multiple papers on chain of thought reasoning. There are tasks that language models struggle with, but when you ask it to explain its reasoning, certain large language models do much better than the scaling for the normal prompt would suggest. Calling this an 'ascribed quality' is crazy, it's just an observation and says nothing about the internals. Hell, you could even test it yourself if you don't trust the papers. Saying that it just looks like P(text|internet) is a tautology, it's a text predictor trained on the internet. This doesn't tell you anything about why phenomena like the above occur, or why it occurs only in large language models and only in some of them.
- benreesman 4y ago
  I hope my comment didn’t imply that I’m anything short of impressed: it’s truly a feat. But it’s one thing to regurgitate a causal structure that’s been said over and over and quite another to propose a novel and falsifiable hypothesis. Telling a novel-ish just so story is well within the reach of a precocious 8 year old, and that’s selling the bright year old short. Let’s see one of things propose a novel hypothesis before we get our drawers moist.
loa_in_ 4y ago
Gpt-3 plays the Wikipedia game pretty well. Ask it to follow and list all articles between one article and other by inter-article links. I liked it because most have Wikipedia corpus already learned.
deleted 4y ago
[deleted]
DebtDeflation 4y ago
The more I play with it, the more it feels like a Google Knowledge Graph result that's been run through a generative model to rewrite the text in a different style.
kewp 4y ago
Isn't it possible that intelligence is P(words|every sequence of words you've ever heard)?
- oliveshell 4y ago
  Possible? Maybe. I believe we know enough now, however, to conclude that it’s astonishingly unlikely.
- Eliezer 4y ago
  Then where would sensible words come from, for the first time, in hominid history?
  - rthomas6 4y ago
    P(sounds when object is present|random sounds other people made when object was present) ?
- joshuahedlund 4y ago
  No, you’re conflating System 1 thinking and System 2 thinking.

chriswarbo 4y ago

> I have yet to see an output from a big language model that doesn’t just look like P(text|internet) True, but the same can be said of many things; e.g. biology just looks like P(reproduction|environment), the economy just looks like P(profit|markets), etc. There can still be rich structure inside, and useful abstractions to describe them.

benreesman 4y ago
Yeah and I hope I didn’t come off like I was trying to knock the technical achievement: it’s remarkable along multiple dissensions: at a minimum technical, infrastructural, mathematical (you don’t throw 25-50k A100s for months at something without running some serious numbers first). It’s possible that I’ve just fallen too far under the influence of Deutsch and Marletto, but as someone who has worked on systems like this I’m rather skeptical that one of these things is going to break the gridlock between quantum theory and general relatively any time soon. There’s no reason why one couldn’t principle but I’ve babysat enough big ML systems that I tend to think in terms of “how do we keep this thing from shitting itself” rather than “damn this thing is going to win a Fields medal if I turn my back on it”.
- sinenomine 4y ago
  It is common to switch to a conservative mode of thought upon entering one's domain of competence; The question is, though, if the expertise and intuition developed during, say, running XGboost classifiers at scale in AdTech really of much relevance when thinking about large transformer models trained with a self-supervised objective and RLHF? If you try to study this in depth, these models can do something the usual "datascience"-tier ones commonly cannot: https://arxiv.org/abs/2205.10343 https://arxiv.org/abs/2205.10343 https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/ https://moultano.wordpress.com/2020/10/18/why-deep-learning-...
  - benreesman 4y ago
    If you’re curious about how we used ensembled weak learners to form a strong enough learner to kick the shit out of everyone except DoubleClick in AdTech, this is a decent start: https://en.m.wikipedia.org/wiki/Ensemble_learning https://en.m.wikipedia.org/wiki/Ensemble_learning. By the standards of AdTech, latent space stuff was pretty unproven when I left the game a few years ago. Low-rank approximations were in some sense implicit in that, but were not at that time an explicit goal. If I were doing an AdTech system from scratch I’d most likely reach for the kind of recommender systems that you’re alluding to sooner. Sounds like you know your stuff :)
sandgiant 4y ago
I don't think anyone looks a biological systems and thinks they are well described by P(reproduction|environment). ChatGPT, on the other hand, is P(text|internet) more or less by definition (it's more than that, but not much more). The fact that we observe what looks like emergent structures such as "understanding", "knowledge" or "reasoning" is fantastic, but it is not in any way incompatible with a P(text|internet) model simply "mimicking" humans. I agree that exploring the inner workings of these emergent features is interesting in it's own right, but all that glitters is not gold.

ImprobableTruth 4y ago

The article links multiple papers on chain of thought reasoning. There are tasks that language models struggle with, but when you ask it to explain its reasoning, certain large language models do much better than the scaling for the normal prompt would suggest. Calling this an 'ascribed quality' is crazy, it's just an observation and says nothing about the internals. Hell, you could even test it yourself if you don't trust the papers. Saying that it just looks like P(text|internet) is a tautology, it's a text predictor trained on the internet. This doesn't tell you anything about why phenomena like the above occur, or why it occurs only in large language models and only in some of them.

benreesman 4y ago
I hope my comment didn’t imply that I’m anything short of impressed: it’s truly a feat. But it’s one thing to regurgitate a causal structure that’s been said over and over and quite another to propose a novel and falsifiable hypothesis. Telling a novel-ish just so story is well within the reach of a precocious 8 year old, and that’s selling the bright year old short. Let’s see one of things propose a novel hypothesis before we get our drawers moist.

loa_in_ 4y ago

Gpt-3 plays the Wikipedia game pretty well. Ask it to follow and list all articles between one article and other by inter-article links. I liked it because most have Wikipedia corpus already learned.

deleted 4y ago

[deleted]

DebtDeflation 4y ago

The more I play with it, the more it feels like a Google Knowledge Graph result that's been run through a generative model to rewrite the text in a different style.

kewp 4y ago

Isn't it possible that intelligence is P(words|every sequence of words you've ever heard)?

oliveshell 4y ago
Possible? Maybe. I believe we know enough now, however, to conclude that it’s astonishingly unlikely.
Eliezer 4y ago
Then where would sensible words come from, for the first time, in hominid history?
- rthomas6 4y ago
  P(sounds when object is present|random sounds other people made when object was present) ?
joshuahedlund 4y ago
No, you’re conflating System 1 thinking and System 2 thinking.