4 ms·

Just because LLMs are limited, doesn't mean they have no real understanding. They do have real understanding, however you may like to define that. The fact that

Just because LLMs are limited, doesn't mean they have no real understanding. They do have real understanding, however you may like to define that. The fact that at a certain point you cannot really argue with them anymore, because they just don't get it, is not very different from humans. As can be observed with this topic.

intended 3y ago
Anthropomorphizing LLMs obscures the issues. There are only tokens being generated. Planning, factualness, logic - these are coincidences that arise because there exists an observer who can interpret the generated tokens. Yes. Humans too can end up in a position where they are regurgitating words without understanding it. The converse, that because humans dont get it, therefore LLMs also understand, doesn't hold. Edit: I too was deeply enamored and tried to create LLM minions for fun and profit. However, it is when you move away from general data to production that the magic is stripped away. It’s prediction, not thinking. The confusion arises because the emergent behaviors are considered human like and we fill in the gaps with human attributes.
- digging 3y ago
  The confusion arises because we don't even know if our own consciousness is purely mathematical or not. We are missing an extremely fundamental link in our chain of understanding, such that it's an open question of whether or not it's even theoretically possible to artificially mimic a human experience. We can see a big gap between the inner working of LLMs (which, unless I'm mistaken, are still not fully understood mechanically) and the inner workings of our brains. But many people assert that this is a qualitative difference, not a quantitative difference. (Probably this is true - we do things like math and spatial perception that seem independent of linguistic abilities. Meaning our brains are more like "LLMs, plus other types of AI mashed together with a central coordination layer") However, nobody on Earth actually knows whether there is a qualitative or only quantitative gap between LLMs and human cognition.
- auggierose 3y ago
  > There are only tokens being generated. Yes, I know how it works. I don't know why it works, though. Until LLMs I had a hard time understanding how humans work, too, but I think LLMs give us great insight into the mechanisms of intelligence. Are they the whole story of intelligence? Hopefully not. Do LLMs teach us anything about consciousness? Certainly that something can appear conscious without being conscious. It could be that I am the only conscious entity here, and you are all just LLMs. If this token generation mechanism is put into a body of flesh, blood, electrons, neurons, will that be a conscious entity? I don't know. If the mechanism has a facility to reason about itself, along, let's say, 3 levels of nested awareness, will that make it conscious? I don't know. I am sure you don't know either.
  - roflyear 3y ago
    The issue is jumping from "I don't know" to "it's alive" or "it has understanding" - some in this thread are making very strong claims regarding consciousness and LLMs, you can't make those claims and then back up your argument with "we can't prove it isn't the case!!!" That isn't exactly how things work.
    - auggierose 3y ago
      Does it have understanding? Yes, it does, without a doubt. If you doubt that, you either haven't used an LLM, or you are in denial. Is it intelligent? Yes, without a doubt. If you doubt that, you either haven't used an LLM, or you are in denial. Is it a limited intelligence? Yes, of course. Is its intelligence comparable to human intelligence? Certainly you can compare it, in some things it is already smarter, in some things it is dumber. Also depends on the human you compare it with. Certain things I don't know, and I've stated them pretty clearly above.
      - roflyear 3y ago
        Define understanding...
        3y ago
- maxdoop 3y ago
  Good grief. So what is thinking, then? Can you prove that humans aren’t just predicting what comes next given a series of inputs ?

intended 3y ago

Anthropomorphizing LLMs obscures the issues. There are only tokens being generated. Planning, factualness, logic - these are coincidences that arise because there exists an observer who can interpret the generated tokens. Yes. Humans too can end up in a position where they are regurgitating words without understanding it. The converse, that because humans dont get it, therefore LLMs also understand, doesn't hold. Edit: I too was deeply enamored and tried to create LLM minions for fun and profit. However, it is when you move away from general data to production that the magic is stripped away. It’s prediction, not thinking. The confusion arises because the emergent behaviors are considered human like and we fill in the gaps with human attributes.

digging 3y ago
The confusion arises because we don't even know if our own consciousness is purely mathematical or not. We are missing an extremely fundamental link in our chain of understanding, such that it's an open question of whether or not it's even theoretically possible to artificially mimic a human experience. We can see a big gap between the inner working of LLMs (which, unless I'm mistaken, are still not fully understood mechanically) and the inner workings of our brains. But many people assert that this is a qualitative difference, not a quantitative difference. (Probably this is true - we do things like math and spatial perception that seem independent of linguistic abilities. Meaning our brains are more like "LLMs, plus other types of AI mashed together with a central coordination layer") However, nobody on Earth actually knows whether there is a qualitative or only quantitative gap between LLMs and human cognition.
auggierose 3y ago
> There are only tokens being generated. Yes, I know how it works. I don't know why it works, though. Until LLMs I had a hard time understanding how humans work, too, but I think LLMs give us great insight into the mechanisms of intelligence. Are they the whole story of intelligence? Hopefully not. Do LLMs teach us anything about consciousness? Certainly that something can appear conscious without being conscious. It could be that I am the only conscious entity here, and you are all just LLMs. If this token generation mechanism is put into a body of flesh, blood, electrons, neurons, will that be a conscious entity? I don't know. If the mechanism has a facility to reason about itself, along, let's say, 3 levels of nested awareness, will that make it conscious? I don't know. I am sure you don't know either.
- roflyear 3y ago
  The issue is jumping from "I don't know" to "it's alive" or "it has understanding" - some in this thread are making very strong claims regarding consciousness and LLMs, you can't make those claims and then back up your argument with "we can't prove it isn't the case!!!" That isn't exactly how things work.
  - auggierose 3y ago
    Does it have understanding? Yes, it does, without a doubt. If you doubt that, you either haven't used an LLM, or you are in denial. Is it intelligent? Yes, without a doubt. If you doubt that, you either haven't used an LLM, or you are in denial. Is it a limited intelligence? Yes, of course. Is its intelligence comparable to human intelligence? Certainly you can compare it, in some things it is already smarter, in some things it is dumber. Also depends on the human you compare it with. Certain things I don't know, and I've stated them pretty clearly above.
    - roflyear 3y ago
      Define understanding...
      - 3y ago
maxdoop 3y ago
Good grief. So what is thinking, then? Can you prove that humans aren’t just predicting what comes next given a series of inputs ?