6 ms·

Until GPT-3 can write something meaningful What is "meaningful"? Honest question. Isn't meaning assigned by a reader? If I'm reading poetry generated by GPT-3

by p1esk 6y ago

Until GPT-3 can write something meaningful What is "meaningful"? Honest question. Isn't meaning assigned by a reader? If I'm reading poetry generated by GPT-3 and I like it just as much as poetry written by a human poet, does it make it meaningful? What if I finetune GPT-3 (or the bigger next gen version) on every scientific paper ever written, and as a result it generates a novel idea that turns out to be valid and useful, should I care if it happened by accident, or without "understanding"? Can the knowledge encoded in the model parameters be interpreted as some form of understanding? If no, why not? What's missing exactly? What is different from how human scientists operate?

Judgmentality 6y ago
Well you can randomly string words together and occasionally get lucky and make a meaningful argument, but that doesn't mean you've created a good method for constructing new ideas. It seems to me the simplest way to decide whether or not something is meaningful (in this context) is whether or not the author (which is the algorithm GPT-3) can respond to criticisms against its own argument in a coherent way. In which case it has to get lucky twice, so it's that much less likely to happen randomly. If the author cannot respond to comments in a comprehensible manner, it's hard to defend the author.
- p1esk 6y ago
  can respond to criticisms against its own argument in a coherent way I'm not sure about GPT-3, but let's imagine GPT-4 next year will be able to do this. It just does not strike me as a particularly high bar to clear. Let's go further, and assume GPT-5 in 2022 will pass the Turing Test (you personally will not be able to tell). What would you say then?
  - inimino 6y ago
    Please don't wildly speculate about technologies you clearly don't understand. "Does not strike me as a particularly high bar" means you're unfamiliar with how these systems work at a deep level (by which I mean, you personally cannot sit down at a terminal and build one). So rather than -ahem- making things up and asking "what then?" -- please ask for textbook recommendations on these topics if you'd like to know more.
    - p1esk 6y ago
      I guess we'll just have to wait and see what happens next year :) p.s. I built my first language model (LSTM based) back in 2014. Then I built a VAE based one. Then a GAN based. None of them were especially good, so I switched to music generation (this actually works pretty well). My most recent project is using sparse transformers for raw audio generation. Building novel DL models is literally in my job description. Please don't wildly speculate about strangers you meet on HN.
      - inimino 6y ago
        Why just wait when you can bet?
        p1esk 6y ago
        In order to bet, we would have to agree on evaluation criteria. A task like "respond to criticisms against its own argument in a coherent way" is difficult to evaluate. Turing test is also pretty vague, and some people already declared it passed many years ago: https://www.bbc.com/news/technology-27762088 https://www.bbc.com/news/technology-27762088
        inimino 6y ago
        Yes, it would be difficult but not impossible to agree on clear criteria. We would also need an impartial third party to make a determination. The Turing test hasn't been passed if I'm the judge. Supposing I were unable to tell the difference between any AI system and a human interlocutor defending the same argument, at any time before 2023, I'd admit my prediction was wholly incorrect. I doubt very much I'll find a taker for this bet, however. The AI field has always been bigger on optimism than results, as we both know.
        p1esk 6y ago
        Yes, I agree TT has not been passed. But probably starting later this year we will be seeing more and more claims it has. At first it will be clear it's not. Then not so clear, and then the goalposts will be moved again, so that when GPT-5 is announced and it is clearly capable of keeping a conversation, the reaction on HN will be the same as the current reaction to GPT-3: "meh".
joe_the_user 6y ago
The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary. The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where paragraph 3 will containment statements that subtly contradict the semantics of paragraph 1. Things like seeing things inside closed drawers and other cues that the thing has no fixed world-model. That's what gives an impression of "meaningless" or "no understanding". (admittedly, I've only played with GPT-2 but this is a description of how texts written by these models at first seem plausible and then implausible as you read more). It's not some philosophical objection akin to "nothing but a human brain can think".
- deleted 6y ago
  [deleted]
- p1esk 6y ago
  the thing has no fixed world-model What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about what's going on inside 175B model to make any dismissive statements about it. I'm also not sure what will happen if you train a much bigger model on much bigger data. Things might start to emerge at some scale.
  - inimino 6y ago
    We know much more about the world-model of GPT-3 (which is none, but it has a language model) than we do about the three year old (which we can't build or reproduce in silico) but we know plenty well enough to say that no, things are not going to magically "emerge" at some scale. You need something fundamentally different from endless cloze deletion exercises to learn what a drawer is.
  - 6y ago
wpietri 6y ago
In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand. There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication. And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate. [1] https://en.wikipedia.org/wiki/Cleromancy https://en.wikipedia.org/wiki/Cleromancy
- p1esk 6y ago
  a process of internal generation of notions How is this "knowledge extraction from data" process different?
  - bregma 6y ago
    Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms. It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.
  - wpietri 6y ago
    Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."
    - p1esk 6y ago

by p1esk 6y ago

Judgmentality 6y ago

Well you can randomly string words together and occasionally get lucky and make a meaningful argument, but that doesn't mean you've created a good method for constructing new ideas. It seems to me the simplest way to decide whether or not something is meaningful (in this context) is whether or not the author (which is the algorithm GPT-3) can respond to criticisms against its own argument in a coherent way. In which case it has to get lucky twice, so it's that much less likely to happen randomly. If the author cannot respond to comments in a comprehensible manner, it's hard to defend the author.

p1esk 6y ago
can respond to criticisms against its own argument in a coherent way I'm not sure about GPT-3, but let's imagine GPT-4 next year will be able to do this. It just does not strike me as a particularly high bar to clear. Let's go further, and assume GPT-5 in 2022 will pass the Turing Test (you personally will not be able to tell). What would you say then?
- inimino 6y ago
  Please don't wildly speculate about technologies you clearly don't understand. "Does not strike me as a particularly high bar" means you're unfamiliar with how these systems work at a deep level (by which I mean, you personally cannot sit down at a terminal and build one). So rather than -ahem- making things up and asking "what then?" -- please ask for textbook recommendations on these topics if you'd like to know more.
  - p1esk 6y ago
    I guess we'll just have to wait and see what happens next year :) p.s. I built my first language model (LSTM based) back in 2014. Then I built a VAE based one. Then a GAN based. None of them were especially good, so I switched to music generation (this actually works pretty well). My most recent project is using sparse transformers for raw audio generation. Building novel DL models is literally in my job description. Please don't wildly speculate about strangers you meet on HN.
    - inimino 6y ago
      Why just wait when you can bet?
      - p1esk 6y ago
        In order to bet, we would have to agree on evaluation criteria. A task like "respond to criticisms against its own argument in a coherent way" is difficult to evaluate. Turing test is also pretty vague, and some people already declared it passed many years ago: https://www.bbc.com/news/technology-27762088 https://www.bbc.com/news/technology-27762088
        inimino 6y ago
        Yes, it would be difficult but not impossible to agree on clear criteria. We would also need an impartial third party to make a determination. The Turing test hasn't been passed if I'm the judge. Supposing I were unable to tell the difference between any AI system and a human interlocutor defending the same argument, at any time before 2023, I'd admit my prediction was wholly incorrect. I doubt very much I'll find a taker for this bet, however. The AI field has always been bigger on optimism than results, as we both know.
        p1esk 6y ago
        Yes, I agree TT has not been passed. But probably starting later this year we will be seeing more and more claims it has. At first it will be clear it's not. Then not so clear, and then the goalposts will be moved again, so that when GPT-5 is announced and it is clearly capable of keeping a conversation, the reaction on HN will be the same as the current reaction to GPT-3: "meh".

joe_the_user 6y ago

The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary. The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where paragraph 3 will containment statements that subtly contradict the semantics of paragraph 1. Things like seeing things inside closed drawers and other cues that the thing has no fixed world-model. That's what gives an impression of "meaningless" or "no understanding". (admittedly, I've only played with GPT-2 but this is a description of how texts written by these models at first seem plausible and then implausible as you read more). It's not some philosophical objection akin to "nothing but a human brain can think".

deleted 6y ago
[deleted]
p1esk 6y ago
the thing has no fixed world-model What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about what's going on inside 175B model to make any dismissive statements about it. I'm also not sure what will happen if you train a much bigger model on much bigger data. Things might start to emerge at some scale.
- inimino 6y ago
  We know much more about the world-model of GPT-3 (which is none, but it has a language model) than we do about the three year old (which we can't build or reproduce in silico) but we know plenty well enough to say that no, things are not going to magically "emerge" at some scale. You need something fundamentally different from endless cloze deletion exercises to learn what a drawer is.
- 6y ago

wpietri 6y ago

In communication, meaning is a collaboration between writer and reader. The writer does their best to convey something; the reader does their best to understand. There's also the kind of meaning that scientists and researchers talk about when they extract knowledge from data. That's pretty different from communication; it's more a process of internal generation of notions and explanations that could later be conveyed in communication. And then there's the meaning that is even more internal. E.g., reading tarot cards or tea leaves, people generate meaning out of nothing. And then there's Pareidolia: https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia If you're saying machine-generated text is meaning in the sense of that last category then sure, it's something we loosely call meaning. But it's fundamentally the same as other sorts of cleromancy [1], just more elaborate. [1] https://en.wikipedia.org/wiki/Cleromancy https://en.wikipedia.org/wiki/Cleromancy

p1esk 6y ago
a process of internal generation of notions How is this "knowledge extraction from data" process different?
- bregma 6y ago
  Immanuel Kant would call the former "rendering a synthetic judgement" and David Hume termed the latter revealing the a priori knowledge. Plato would call both simply giving substance to the forms. It seems none of this is something new. It's just that you no longer need a good education to learn about old ideas; you can come up with them on your own.
- wpietri 6y ago
  Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."
  - p1esk 6y ago