9 ms·

> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind,

by tines 4y ago

> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning. etc. I did an experiment recently where I asked ChatGPT to "tell me an idea [you] have never heard before". ChatGPT replied with what sounded like an idea for a startup, which was delivering farm-fresh vegetables to customers' doors. This is of course not an idea it has never heard before, it's on the internet. If you asked a human this, they would give you an idea they had never heard before, whereas ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. (Obviously a gross simplification of the algorithm but the point stands.) This is a difference between ChatGPT's algorithm and human reasoning. The things that you mention, the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Am I wrong? I'm open to persuasion.

fl7305 4y ago
> the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Is that a falsifiable claim? What would an empirical test look like that would determine whether the claim ("GPT-4 does not have a model of the world") is correct or not?
- tines 4y ago
  I guess ChatGPT having a model of the world is not something I'd argue against, my mistake. I do think the data it contains represents a quite nuanced model of the world. But I don't think this is a very strong claim. A geographical map has a model of the world too.
  - fl7305 4y ago
    Thanks. To get to what I think is the core of your argument (?) > ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. I can definitely see a scenario where we manage to build an ultra-intelligent machine that can figure out any logical puzzle we put to it, but where it still isn't "creative" in any way. But GPT-4 seems to be not just intelligent, but also creative. Have you seen the "confetti has left the cannon" example? For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new phrase that has the same meaning, which is completely new? chatgpt: Certainly! Here's a new phrase with a similar meaning: "The confetti has left the cannon." This phrase suggests that once something is out in the open or has happened, it's impossible to reverse or contain it, much like confetti that has been shot out of a cannon during a celebration. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683
    - tines 4y ago
      That does seem really impressive. But don't you think that it's pretty likely that this, or something phrased slightly differently, appeared in the training data?
      - fl7305 4y ago
        > But don't you think that it's pretty likely that this, or something phrased slightly differently, appeared in the training data? I don't think so, but I could be wrong. It's definitely not "likely", see the math below. I base that on the fact that people seemed to spend quite a bit of time trying to find the phrase "the confetti has left the cannon" that GPT-4 phrased. It seems Google search has no records of it before then? I've seen many other examples where GPT-4 can translate sentences between using different types of idioms, and I just can't picture all these weird examples already being present on the Internet? Do you think GPT-4 is a stochastic parrot that just has a large database of responses? If so, how would we test that claim? What logical and reasoning problems can we give it where it fails to answer, but a human doesn't? My understanding is that even with an extremely limited vocabulary of 32 words, you quickly run out of atoms in the universe (10^80) if you string more than 50 words together. If your vocabulary instead is 10k words, you reach 10^80 combinations after 20 words. By training the LLMs on "fill in the missing word", they were forced to evolve ever more sophisticated algorithms. If you look at the performance over the last 5 years of increasingly larger LLMs, there was a hockey-stick jump in performance 1-2 years ago. My hunch is that is when they started evolving structures to generate better responses by using logic and reasoning instead of lookup tables.
        vertebrate 4y ago
        > I base that on the fact that people seemed to spend quite a bit of time trying to find the phrase "the confetti has left the cannon" that GPT-4 phrased. It seems Google search has no records of it before then? Could it be that the expression in some form has been used in languages other than English?
        4y ago
noahtallen 4y ago
I think it's certainly fair to say that GPT's "reasoning" is different from human reasoning. But I think the core debate we're having is whether the difference really matters in some situations. Certainly, Midjourney's "creativity" is different from human creativity. But it is producing results that we marvel at. It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. And I think many situations are like that. We can always say that human creativity/reasoning/x will always be different from artificial reasoning. But even today, GPT's statistical model replicates many aspects of human reasoning virtually. Is that really an illusion (implying its fake and potentially useless), or is it just a different way of achieving a similar result? Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? At the end of the day, some of these discussions feel like bike shedding about what words like "reasoning" mean philosophically. But what will ultimately matter is how well these models perform at real world tasks, and what impact that will have on humanity. It doesn't really matter if it's virtualized reasoning or "real" human reasoning at that point.
- tines 4y ago
  > It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. Absolutely, and I hope none of my comments are taken in a way that disparages how amazing ChatGPT and Stable Diffusion et al. are. I'm just debating how humanlike they are. > Is that really an illusion (implying its fake and potentially useless) I don't think that because it's an illusion means that its useless. Magnets look like telekinesis, but that effect being an illusion doesn't mean that magnets are useless; far from it, and once we admit that they are what they are, they become even more useful. > Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? It only matters if people are trying to say that ChatGPT is essentially human, that idea is all I was replying to. I completely agree with you here.
brazzy 4y ago
Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial. And remixing stuff it has heard before is exactly what ChatGPT is doing. What it sucks at is the "feels like it's new" part, but fundamentally it would be quite easily capable of creating output that combines concepts with maximally negative correlation, the only thing that's truly missing is the ability to interpret the prompt as an instruction to do that.
- pulvinar 4y ago
  Certainly. I mean we've seen all 26 letters before-- ChatGPT is just remixing them. How does one actually measure novelty, without having to know everything first?
  - brazzy 4y ago
    The entire strength of large language models like GPT is that they do know a frighteningly good approximation of everything, in terms of having been trained on text written about it.
- fl7305 4y ago
  > And remixing stuff it has heard before is exactly what ChatGPT is doing. Check out my "the confetti has left the cannon" example above. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683 Maybe still "remixing", but it sure feels like new to me.
tornato7 4y ago
For what it's worth, I asked ChatGPT to come up with an original idea but I generated 25 random English words and told it to use them as inspiration. This can help the output be more creative and original. Here's what it came up with: > The BerryPulse is an innovative, eco-friendly device that captures the energy released during the natural decomposition of berries to produce heat and electricity. The device comprises a closed container, where a cluster of berries is placed in a specially designed compartment, acting as an organic fuel source. The decomposition process is intensified using a biodegradable catalyst, which accelerates the natural breakdown of the berries while ensuring minimal environmental impact.

by tines 4y ago

fl7305 4y ago

> the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Is that a falsifiable claim? What would an empirical test look like that would determine whether the claim ("GPT-4 does not have a model of the world") is correct or not?

tines 4y ago
I guess ChatGPT having a model of the world is not something I'd argue against, my mistake. I do think the data it contains represents a quite nuanced model of the world. But I don't think this is a very strong claim. A geographical map has a model of the world too.
- fl7305 4y ago
  Thanks. To get to what I think is the core of your argument (?) > ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. I can definitely see a scenario where we manage to build an ultra-intelligent machine that can figure out any logical puzzle we put to it, but where it still isn't "creative" in any way. But GPT-4 seems to be not just intelligent, but also creative. Have you seen the "confetti has left the cannon" example? For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new phrase that has the same meaning, which is completely new? chatgpt: Certainly! Here's a new phrase with a similar meaning: "The confetti has left the cannon." This phrase suggests that once something is out in the open or has happened, it's impossible to reverse or contain it, much like confetti that has been shot out of a cannon during a celebration. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683
  - tines 4y ago
    That does seem really impressive. But don't you think that it's pretty likely that this, or something phrased slightly differently, appeared in the training data?
    - fl7305 4y ago
      > But don't you think that it's pretty likely that this, or something phrased slightly differently, appeared in the training data? I don't think so, but I could be wrong. It's definitely not "likely", see the math below. I base that on the fact that people seemed to spend quite a bit of time trying to find the phrase "the confetti has left the cannon" that GPT-4 phrased. It seems Google search has no records of it before then? I've seen many other examples where GPT-4 can translate sentences between using different types of idioms, and I just can't picture all these weird examples already being present on the Internet? Do you think GPT-4 is a stochastic parrot that just has a large database of responses? If so, how would we test that claim? What logical and reasoning problems can we give it where it fails to answer, but a human doesn't? My understanding is that even with an extremely limited vocabulary of 32 words, you quickly run out of atoms in the universe (10^80) if you string more than 50 words together. If your vocabulary instead is 10k words, you reach 10^80 combinations after 20 words. By training the LLMs on "fill in the missing word", they were forced to evolve ever more sophisticated algorithms. If you look at the performance over the last 5 years of increasingly larger LLMs, there was a hockey-stick jump in performance 1-2 years ago. My hunch is that is when they started evolving structures to generate better responses by using logic and reasoning instead of lookup tables.
      - vertebrate 4y ago
        > I base that on the fact that people seemed to spend quite a bit of time trying to find the phrase "the confetti has left the cannon" that GPT-4 phrased. It seems Google search has no records of it before then? Could it be that the expression in some form has been used in languages other than English?
        4y ago

noahtallen 4y ago

I think it's certainly fair to say that GPT's "reasoning" is different from human reasoning. But I think the core debate we're having is whether the difference really matters in some situations. Certainly, Midjourney's "creativity" is different from human creativity. But it is producing results that we marvel at. It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. And I think many situations are like that. We can always say that human creativity/reasoning/x will always be different from artificial reasoning. But even today, GPT's statistical model replicates many aspects of human reasoning virtually. Is that really an illusion (implying its fake and potentially useless), or is it just a different way of achieving a similar result? Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? At the end of the day, some of these discussions feel like bike shedding about what words like "reasoning" mean philosophically. But what will ultimately matter is how well these models perform at real world tasks, and what impact that will have on humanity. It doesn't really matter if it's virtualized reasoning or "real" human reasoning at that point.

tines 4y ago
> It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. Absolutely, and I hope none of my comments are taken in a way that disparages how amazing ChatGPT and Stable Diffusion et al. are. I'm just debating how humanlike they are. > Is that really an illusion (implying its fake and potentially useless) I don't think that because it's an illusion means that its useless. Magnets look like telekinesis, but that effect being an illusion doesn't mean that magnets are useless; far from it, and once we admit that they are what they are, they become even more useful. > Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? It only matters if people are trying to say that ChatGPT is essentially human, that idea is all I was replying to. I completely agree with you here.

brazzy 4y ago

Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial. And remixing stuff it has heard before is exactly what ChatGPT is doing. What it sucks at is the "feels like it's new" part, but fundamentally it would be quite easily capable of creating output that combines concepts with maximally negative correlation, the only thing that's truly missing is the ability to interpret the prompt as an instruction to do that.

pulvinar 4y ago
Certainly. I mean we've seen all 26 letters before-- ChatGPT is just remixing them. How does one actually measure novelty, without having to know everything first?
- brazzy 4y ago
  The entire strength of large language models like GPT is that they do know a frighteningly good approximation of everything, in terms of having been trained on text written about it.
fl7305 4y ago
> And remixing stuff it has heard before is exactly what ChatGPT is doing. Check out my "the confetti has left the cannon" example above. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683 Maybe still "remixing", but it sure feels like new to me.

tornato7 4y ago

For what it's worth, I asked ChatGPT to come up with an original idea but I generated 25 random English words and told it to use them as inspiration. This can help the output be more creative and original. Here's what it came up with: > The BerryPulse is an innovative, eco-friendly device that captures the energy released during the natural decomposition of berries to produce heat and electricity. The device comprises a closed container, where a cluster of berries is placed in a specially designed compartment, acting as an organic fuel source. The decomposition process is intensified using a biodegradable catalyst, which accelerates the natural breakdown of the berries while ensuring minimal environmental impact.