5 ms·

It's all about emergent complexity. While you can reduce it to "just" statistical auto-completion of the next word, we are seeing evidence of abstraction and

It's all about emergent complexity. While you can reduce it to "just" statistical auto-completion of the next word, we are seeing evidence of abstraction and reasoning produced as a higher-order effect of these simple completions. It's a bit like the Sagan quote: "If you wish to make an apple pie from scratch, you must first invent the universe". Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning, etc. Because the models are opaque, we can't yet point to a certain batch of CPU cycles and say "there! it just engaged in abstract reasoning". But we can see from the output that to some extent it's happening, somehow. We also see effects like this when looking at collective intelligence of bees and ants. While each individual insect is only performing simple actions with extremely limited cognitive processing, it can add up to highly complex and intelligent/adaptive mechanics at the level of the swarm. There are many phenomena like this in nature.

tines 4y ago
> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning. etc. I did an experiment recently where I asked ChatGPT to "tell me an idea [you] have never heard before". ChatGPT replied with what sounded like an idea for a startup, which was delivering farm-fresh vegetables to customers' doors. This is of course not an idea it has never heard before, it's on the internet. If you asked a human this, they would give you an idea they had never heard before, whereas ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. (Obviously a gross simplification of the algorithm but the point stands.) This is a difference between ChatGPT's algorithm and human reasoning. The things that you mention, the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Am I wrong? I'm open to persuasion.
- fl7305 4y ago
  > the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Is that a falsifiable claim? What would an empirical test look like that would determine whether the claim ("GPT-4 does not have a model of the world") is correct or not?
  - tines 4y ago
    I guess ChatGPT having a model of the world is not something I'd argue against, my mistake. I do think the data it contains represents a quite nuanced model of the world. But I don't think this is a very strong claim. A geographical map has a model of the world too.
    - fl7305 4y ago
      Thanks. To get to what I think is the core of your argument (?) > ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. I can definitely see a scenario where we manage to build an ultra-intelligent machine that can figure out any logical puzzle we put to it, but where it still isn't "creative" in any way. But GPT-4 seems to be not just intelligent, but also creative. Have you seen the "confetti has left the cannon" example? For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new phrase that has the same meaning, which is completely new? chatgpt: Certainly! Here's a new phrase with a similar meaning: "The confetti has left the cannon." This phrase suggests that once something is out in the open or has happened, it's impossible to reverse or contain it, much like confetti that has been shot out of a cannon during a celebration. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683
      - tines 4y ago
- noahtallen 4y ago
  I think it's certainly fair to say that GPT's "reasoning" is different from human reasoning. But I think the core debate we're having is whether the difference really matters in some situations. Certainly, Midjourney's "creativity" is different from human creativity. But it is producing results that we marvel at. It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. And I think many situations are like that. We can always say that human creativity/reasoning/x will always be different from artificial reasoning. But even today, GPT's statistical model replicates many aspects of human reasoning virtually. Is that really an illusion (implying its fake and potentially useless), or is it just a different way of achieving a similar result? Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? At the end of the day, some of these discussions feel like bike shedding about what words like "reasoning" mean philosophically. But what will ultimately matter is how well these models perform at real world tasks, and what impact that will have on humanity. It doesn't really matter if it's virtualized reasoning or "real" human reasoning at that point.
  - tines 4y ago
    > It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. Absolutely, and I hope none of my comments are taken in a way that disparages how amazing ChatGPT and Stable Diffusion et al. are. I'm just debating how humanlike they are. > Is that really an illusion (implying its fake and potentially useless) I don't think that because it's an illusion means that its useless. Magnets look like telekinesis, but that effect being an illusion doesn't mean that magnets are useless; far from it, and once we admit that they are what they are, they become even more useful. > Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? It only matters if people are trying to say that ChatGPT is essentially human, that idea is all I was replying to. I completely agree with you here.
- brazzy 4y ago
  Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial. And remixing stuff it has heard before is exactly what ChatGPT is doing. What it sucks at is the "feels like it's new" part, but fundamentally it would be quite easily capable of creating output that combines concepts with maximally negative correlation, the only thing that's truly missing is the ability to interpret the prompt as an instruction to do that.
  - pulvinar 4y ago
    Certainly. I mean we've seen all 26 letters before-- ChatGPT is just remixing them. How does one actually measure novelty, without having to know everything first?
    - brazzy 4y ago
      The entire strength of large language models like GPT is that they do know a frighteningly good approximation of everything, in terms of having been trained on text written about it.
  - fl7305 4y ago
    > And remixing stuff it has heard before is exactly what ChatGPT is doing. Check out my "the confetti has left the cannon" example above. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683 Maybe still "remixing", but it sure feels like new to me.
- tornato7 4y ago
  For what it's worth, I asked ChatGPT to come up with an original idea but I generated 25 random English words and told it to use them as inspiration. This can help the output be more creative and original. Here's what it came up with: > The BerryPulse is an innovative, eco-friendly device that captures the energy released during the natural decomposition of berries to produce heat and electricity. The device comprises a closed container, where a cluster of berries is placed in a specially designed compartment, acting as an organic fuel source. The decomposition process is intensified using a biodegradable catalyst, which accelerates the natural breakdown of the berries while ensuring minimal environmental impact.
ttpphd 4y ago
This is really lofty language without much evidence to back it up. It fluffs up techie people and makes them feel powerful, but it doesn't really describe large language models nor does it describe linguistic processes.
- danenania 4y ago
  The evidence is ChatGPT's output. Unless you're saying that passing the bar exam, writing working code, etc. doesn't require abstract reasoning abilities or a model of the world?
  - ttpphd 4y ago
    It's a large language model. It is fed training data. It is not that impressive when it spits out stuff that looks like its training data. You are the one asserting things without evidence.
    - danenania 4y ago
      It can pass tests and exams with answers that were not included in its training corpus. For example, it passed the 2023 unified bar exam, though its training cut off in 2021. Yes, it can look at previous test questions and answers, just like human law students can. Are you therefore claiming that human law students don't engage in abstract reasoning when they take the bar exam, since they studied with tests from previous years? It can also write code for novel use cases that have never been done before. I gave it a task like this a few days ago and it got it right on the first try. There are literally millions of empirical data points that contradict you.
      - ttpphd 4y ago
        It is a large language model. It manipulates text based on context and the imprint of its vast training. You are not able to articulate a theory of reasoning. You are just pointing to the output of an algorithm and saying "this must mean something!" There isn't even a working model of reasoning here, it's just a human being impressed that a tool for manipulating symbols is able to manipulate symbols after training it to manipulate symbols in the specific way that you want symbols manipulated. Where is your articulated theory of abstract reasoning?
squeaky-clean 4y ago
Turns out good usage of "language" requires a model of the world in which that language exists. "The purple, two eyed, green, five eyed, invisible frog said moo" is a grammatically fine sentence. But logically it makes no sense, does it have two eyes or five? Is it green or purple or invisible? Frogs don't typically say moo. To have actual coherent usage of language, you need a model of the world. Not just the world, but the current domain you're using language in. "The frog brainwashed the crowd with its psychic powers" is nonsense in a biology paper, but perfectly valid inside of the cartoon Futurama. In ChatGPT the language-model and world-model are really just the same model, which makes a lot of sense.
- danenania 4y ago
  Very well said. We think of a word as "just" a word, a simple, primarily textual thing, but it's actually a vertex on an extremely large and complex many-dimensional graph that includes connections related to meaning, logic/reasoning, knowledge about reality, emotional sentiment, and so on. The literal textual representation of the word--the letters it consists of--are just one property among many, and probably one of the least important to producing sensible output. GPT is discovering the shape of this super-graph and learning to navigate its connections.

tines 4y ago

> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning. etc. I did an experiment recently where I asked ChatGPT to "tell me an idea [you] have never heard before". ChatGPT replied with what sounded like an idea for a startup, which was delivering farm-fresh vegetables to customers' doors. This is of course not an idea it has never heard before, it's on the internet. If you asked a human this, they would give you an idea they had never heard before, whereas ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. (Obviously a gross simplification of the algorithm but the point stands.) This is a difference between ChatGPT's algorithm and human reasoning. The things that you mention, the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Am I wrong? I'm open to persuasion.

fl7305 4y ago
> the model of the world, theory of mind, etc. are statistical illusions which have observable differences from the real thing. Is that a falsifiable claim? What would an empirical test look like that would determine whether the claim ("GPT-4 does not have a model of the world") is correct or not?
- tines 4y ago
  I guess ChatGPT having a model of the world is not something I'd argue against, my mistake. I do think the data it contains represents a quite nuanced model of the world. But I don't think this is a very strong claim. A geographical map has a model of the world too.
  - fl7305 4y ago
    Thanks. To get to what I think is the core of your argument (?) > ChatGPT simply "finds" training data where someone asked a similar question, and produces the likely response, which is an idea that it has actually "heard," or seen in its training data, before. I can definitely see a scenario where we manage to build an ultra-intelligent machine that can figure out any logical puzzle we put to it, but where it still isn't "creative" in any way. But GPT-4 seems to be not just intelligent, but also creative. Have you seen the "confetti has left the cannon" example? For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new phrase that has the same meaning, which is completely new? chatgpt: Certainly! Here's a new phrase with a similar meaning: "The confetti has left the cannon." This phrase suggests that once something is out in the open or has happened, it's impossible to reverse or contain it, much like confetti that has been shot out of a cannon during a celebration. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683
    - tines 4y ago
noahtallen 4y ago
I think it's certainly fair to say that GPT's "reasoning" is different from human reasoning. But I think the core debate we're having is whether the difference really matters in some situations. Certainly, Midjourney's "creativity" is different from human creativity. But it is producing results that we marvel at. It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. And I think many situations are like that. We can always say that human creativity/reasoning/x will always be different from artificial reasoning. But even today, GPT's statistical model replicates many aspects of human reasoning virtually. Is that really an illusion (implying its fake and potentially useless), or is it just a different way of achieving a similar result? Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? At the end of the day, some of these discussions feel like bike shedding about what words like "reasoning" mean philosophically. But what will ultimately matter is how well these models perform at real world tasks, and what impact that will have on humanity. It doesn't really matter if it's virtualized reasoning or "real" human reasoning at that point.
- tines 4y ago
  > It's creative not because it's doing the exact same philosophical thing humans do, but because it can produce the same effect. Absolutely, and I hope none of my comments are taken in a way that disparages how amazing ChatGPT and Stable Diffusion et al. are. I'm just debating how humanlike they are. > Is that really an illusion (implying its fake and potentially useless) I don't think that because it's an illusion means that its useless. Magnets look like telekinesis, but that effect being an illusion doesn't mean that magnets are useless; far from it, and once we admit that they are what they are, they become even more useful. > Plus, different models will excel at different thing. GPT's model will excel at synthesizing answers from far more information than a single human will ever be able to know. Does it really matter if it's not identical to human reasoning on a philosophical or biological level, if it can do things humans can't do? It only matters if people are trying to say that ChatGPT is essentially human, that idea is all I was replying to. I completely agree with you here.
brazzy 4y ago
Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial. And remixing stuff it has heard before is exactly what ChatGPT is doing. What it sucks at is the "feels like it's new" part, but fundamentally it would be quite easily capable of creating output that combines concepts with maximally negative correlation, the only thing that's truly missing is the ability to interpret the prompt as an instruction to do that.
- pulvinar 4y ago
  Certainly. I mean we've seen all 26 letters before-- ChatGPT is just remixing them. How does one actually measure novelty, without having to know everything first?
  - brazzy 4y ago
    The entire strength of large language models like GPT is that they do know a frighteningly good approximation of everything, in terms of having been trained on text written about it.
- fl7305 4y ago
  > And remixing stuff it has heard before is exactly what ChatGPT is doing. Check out my "the confetti has left the cannon" example above. https://news.ycombinator.com/item?id=35346683 https://news.ycombinator.com/item?id=35346683 Maybe still "remixing", but it sure feels like new to me.
tornato7 4y ago
For what it's worth, I asked ChatGPT to come up with an original idea but I generated 25 random English words and told it to use them as inspiration. This can help the output be more creative and original. Here's what it came up with: > The BerryPulse is an innovative, eco-friendly device that captures the energy released during the natural decomposition of berries to produce heat and electricity. The device comprises a closed container, where a cluster of berries is placed in a specially designed compartment, acting as an organic fuel source. The decomposition process is intensified using a biodegradable catalyst, which accelerates the natural breakdown of the berries while ensuring minimal environmental impact.

ttpphd 4y ago

This is really lofty language without much evidence to back it up. It fluffs up techie people and makes them feel powerful, but it doesn't really describe large language models nor does it describe linguistic processes.

danenania 4y ago
The evidence is ChatGPT's output. Unless you're saying that passing the bar exam, writing working code, etc. doesn't require abstract reasoning abilities or a model of the world?
- ttpphd 4y ago
  It's a large language model. It is fed training data. It is not that impressive when it spits out stuff that looks like its training data. You are the one asserting things without evidence.
  - danenania 4y ago
    It can pass tests and exams with answers that were not included in its training corpus. For example, it passed the 2023 unified bar exam, though its training cut off in 2021. Yes, it can look at previous test questions and answers, just like human law students can. Are you therefore claiming that human law students don't engage in abstract reasoning when they take the bar exam, since they studied with tests from previous years? It can also write code for novel use cases that have never been done before. I gave it a task like this a few days ago and it got it right on the first try. There are literally millions of empirical data points that contradict you.
    - ttpphd 4y ago
      It is a large language model. It manipulates text based on context and the imprint of its vast training. You are not able to articulate a theory of reasoning. You are just pointing to the output of an algorithm and saying "this must mean something!" There isn't even a working model of reasoning here, it's just a human being impressed that a tool for manipulating symbols is able to manipulate symbols after training it to manipulate symbols in the specific way that you want symbols manipulated. Where is your articulated theory of abstract reasoning?

squeaky-clean 4y ago

Turns out good usage of "language" requires a model of the world in which that language exists. "The purple, two eyed, green, five eyed, invisible frog said moo" is a grammatically fine sentence. But logically it makes no sense, does it have two eyes or five? Is it green or purple or invisible? Frogs don't typically say moo. To have actual coherent usage of language, you need a model of the world. Not just the world, but the current domain you're using language in. "The frog brainwashed the crowd with its psychic powers" is nonsense in a biology paper, but perfectly valid inside of the cartoon Futurama. In ChatGPT the language-model and world-model are really just the same model, which makes a lot of sense.

danenania 4y ago
Very well said. We think of a word as "just" a word, a simple, primarily textual thing, but it's actually a vertex on an extremely large and complex many-dimensional graph that includes connections related to meaning, logic/reasoning, knowledge about reality, emotional sentiment, and so on. The literal textual representation of the word--the letters it consists of--are just one property among many, and probably one of the least important to producing sensible output. GPT is discovering the shape of this super-graph and learning to navigate its connections.