7 ms·

Define reason before making grandiose claims.

quantadev 2y ago
I think "reasoning" is the best word in the English language that I know of to describe what LLMs are doing. Lots of people disagree because for about 2000 years that word always implied consciousness, and LLMs definitely don't have consciousness.
- gaganyaan 2y ago
  Define consciousness before making that claim either.
  - quantadev 2y ago
    I'm fine with the standard dictionary definition in this case. :P
    - gaganyaan 2y ago
      There's as many definitions as there are dictionaries, but let's try picking "awareness". LLMs are aware of many things, so they are conscious of those things, meaning they display some level of consciousness.
      - quantadev 2y ago
        Nice one. Now do "qualia"
        gaganyaan 2y ago
        Qualia is a silly god of the gaps style argument. There's no difference between "seeing red" and "having your neurons tickled so that you think you're seeing red". People really don't understand consciousness, but it's actually quite simple when you realize that the question isn't "is this conscious?", it's "what is this conscious of?". A rock is not conscious of anything, because it has no state that represents anything. A sunflower is conscious of the sun's position, because it has state that represents that. A cat is conscious of much more, such as "this mouse is food". An LLM is conscious of much more, but less than a human. The LLM is a model that is conscious of many things, as experienced through the medium of human text.
        quantadev 2y ago
        The fact that we can tell even from an EEG (a grossly inaccurate measurement of brain activity) whether a person is conscious or unconscious, indicates there's a physical reality to it, which might be able to be understood by science some day. I think Qualia is a wave phenomena, and all wave phenomena simultaneously can be described as "existing" and "not existing" as follows: Think of a football stadium "wave". It has a beginning, ending, location, velocity, etc., all being scientifically measurable observables. Yet I ask you "Does the wave exist, or is it merely people in various states of arm positions?" There is no right or wrong answer. This is the nature of waves. So if you want to argue consciousness "does not exist" I say fine, that is a valid "perspective", just like the denial of the existence of any other wave phenomena is a valid perspective. However keep in mind also that the entire universe is also made of waves.
        gaganyaan 2y ago
        Sure, waves exist, whatever. Not to be rude, but so what? This kind of comes across as "quantum waves therefore the Universe Is Love" or something. What about the waves matters here? I'm not arguing that consciousness doesn't exist, I'm saying that it's not a binary thing, and trying to provide a useful, falsifiable definition of it that provides a way to actually communicate about it instead of talking past each other.
        quantadev 2y ago
        I partially answered this on the other post. Some people think Qualia is emergent purely from computation. Some people (like me) think there's a different and very specific and very unique thing going on inside brains that is responsible for Qualia, and I think to understand it scientifically it's going to be a "Wave Theory". I'm not even pushing the panpsychism view that "Everything has Consciousness, because all atoms have EM fields" at all, but when you get right down to it, EM fields are more "real" than a computation is. A computation is purely theoretical. We can build computer logic circuits out of water pipes or dominoes. So it's just obvious to me Qualia is not built on computations. Believing Qualia is made of computations is more of a naieve "Universe is Love" type position than believing that Qualia is built of something REAL like EM fields. Calculations are not even real. For example the number 0100101110110 in a computer is purely a "label" for something real. Numbers in computers are just labels. EM waves are the actual thing, and actually real. The Map is not the Territory. Computers are Maps. Qualia is a Territory.
- HeatrayEnjoyer 2y ago
  It's not possible to know if anyone has consciousness, other than them themselves. We must be *very* careful before making authoritative statements about who and what has qualia.
  - quantadev 2y ago
    [flagged]
    - HeatrayEnjoyer 2y ago
      [flagged]
      - deleted 2y ago
        [deleted]
      - quantadev 2y ago
        [flagged]
  - godelski 2y ago
    The logic goes the other way. The claim that is extraordinary is that a rock with electricity has consciousness. Nothing else we've built has shown evidence of consciousness and there's good explanations for the results we see without needing consciousness. Sure, it's still debatable for us humans (the term isn't well defined) but that doesn't change the direction of required proof.
    - mistermann 2y ago
      Any assertion has a burden of proof, you are likely referring to cultural/colloquial norms/beliefs on the matter. There is cultural/normative truth, and then there are various stricter/technical interpretations (using forms of logic other than binary, which causes reality to appear very differently), and making such distinctions is typically culturally "discouraged" (it is rude, pedantic, not what HN is for, etc).
Fellshard 2y ago
The grandiose claim would be that LLMs have reason to begin with. It is quite normal for things to not have reasoning capacity.
- gaganyaan 2y ago
  They can be trivially shown to reason. It's quite normal for many things to have reasoning capacity, including amoeba, sunflowers, cats, LLMs, and humans. The real issue is establishing the limitations of their reasoning capabilities, and how to improve them.
  - Fellshard 2y ago
    As has been noted in other threads, for a low enough definition of reason, yes, certainly. But even an amoeba appears to reason better than an LLM, which only regurgitates by nature. There is no independent cognition whatsoever, no logical constructs and decisions, no abstractions. Reducing reasoning beings to the level of AI is an affront to—and demonstrates a genuine lack of understanding of—the nature of organisms and reason itself, and the nature of AI and its capabilites.

I think "reasoning" is the best word in the English language that I know of to describe what LLMs are doing. Lots of people disagree because for about 2000 years that word always implied consciousness, and LLMs definitely don't have consciousness.

gaganyaan 2y ago
Define consciousness before making that claim either.
- quantadev 2y ago
  I'm fine with the standard dictionary definition in this case. :P
  - gaganyaan 2y ago
    There's as many definitions as there are dictionaries, but let's try picking "awareness". LLMs are aware of many things, so they are conscious of those things, meaning they display some level of consciousness.
    - quantadev 2y ago
      Nice one. Now do "qualia"
      - gaganyaan 2y ago
        Qualia is a silly god of the gaps style argument. There's no difference between "seeing red" and "having your neurons tickled so that you think you're seeing red". People really don't understand consciousness, but it's actually quite simple when you realize that the question isn't "is this conscious?", it's "what is this conscious of?". A rock is not conscious of anything, because it has no state that represents anything. A sunflower is conscious of the sun's position, because it has state that represents that. A cat is conscious of much more, such as "this mouse is food". An LLM is conscious of much more, but less than a human. The LLM is a model that is conscious of many things, as experienced through the medium of human text.
        quantadev 2y ago
        The fact that we can tell even from an EEG (a grossly inaccurate measurement of brain activity) whether a person is conscious or unconscious, indicates there's a physical reality to it, which might be able to be understood by science some day. I think Qualia is a wave phenomena, and all wave phenomena simultaneously can be described as "existing" and "not existing" as follows: Think of a football stadium "wave". It has a beginning, ending, location, velocity, etc., all being scientifically measurable observables. Yet I ask you "Does the wave exist, or is it merely people in various states of arm positions?" There is no right or wrong answer. This is the nature of waves. So if you want to argue consciousness "does not exist" I say fine, that is a valid "perspective", just like the denial of the existence of any other wave phenomena is a valid perspective. However keep in mind also that the entire universe is also made of waves.
        gaganyaan 2y ago
        Sure, waves exist, whatever. Not to be rude, but so what? This kind of comes across as "quantum waves therefore the Universe Is Love" or something. What about the waves matters here? I'm not arguing that consciousness doesn't exist, I'm saying that it's not a binary thing, and trying to provide a useful, falsifiable definition of it that provides a way to actually communicate about it instead of talking past each other.
        quantadev 2y ago
        I partially answered this on the other post. Some people think Qualia is emergent purely from computation. Some people (like me) think there's a different and very specific and very unique thing going on inside brains that is responsible for Qualia, and I think to understand it scientifically it's going to be a "Wave Theory". I'm not even pushing the panpsychism view that "Everything has Consciousness, because all atoms have EM fields" at all, but when you get right down to it, EM fields are more "real" than a computation is. A computation is purely theoretical. We can build computer logic circuits out of water pipes or dominoes. So it's just obvious to me Qualia is not built on computations. Believing Qualia is made of computations is more of a naieve "Universe is Love" type position than believing that Qualia is built of something REAL like EM fields. Calculations are not even real. For example the number 0100101110110 in a computer is purely a "label" for something real. Numbers in computers are just labels. EM waves are the actual thing, and actually real. The Map is not the Territory. Computers are Maps. Qualia is a Territory.
HeatrayEnjoyer 2y ago
It's not possible to know if anyone has consciousness, other than them themselves. We must be *very* careful before making authoritative statements about who and what has qualia.
- quantadev 2y ago
  [flagged]
  - HeatrayEnjoyer 2y ago
    [flagged]
    - deleted 2y ago
      [deleted]
    - quantadev 2y ago
      [flagged]
- godelski 2y ago
  The logic goes the other way. The claim that is extraordinary is that a rock with electricity has consciousness. Nothing else we've built has shown evidence of consciousness and there's good explanations for the results we see without needing consciousness. Sure, it's still debatable for us humans (the term isn't well defined) but that doesn't change the direction of required proof.
  - mistermann 2y ago
    Any assertion has a burden of proof, you are likely referring to cultural/colloquial norms/beliefs on the matter. There is cultural/normative truth, and then there are various stricter/technical interpretations (using forms of logic other than binary, which causes reality to appear very differently), and making such distinctions is typically culturally "discouraged" (it is rude, pedantic, not what HN is for, etc).

Fellshard 2y ago

The grandiose claim would be that LLMs have reason to begin with. It is quite normal for things to not have reasoning capacity.

gaganyaan 2y ago
They can be trivially shown to reason. It's quite normal for many things to have reasoning capacity, including amoeba, sunflowers, cats, LLMs, and humans. The real issue is establishing the limitations of their reasoning capabilities, and how to improve them.
- Fellshard 2y ago
  As has been noted in other threads, for a low enough definition of reason, yes, certainly. But even an amoeba appears to reason better than an LLM, which only regurgitates by nature. There is no independent cognition whatsoever, no logical constructs and decisions, no abstractions. Reducing reasoning beings to the level of AI is an affront to—and demonstrates a genuine lack of understanding of—the nature of organisms and reason itself, and the nature of AI and its capabilites.