5 ms·

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known

LLMs lack qualia, among other things. If I ask an LLM "what is an apple?" it tells me: > An apple is the edible fruit of the apple tree, scientifically known as Malus domestica. It is one of the world's most widely grown fruits and is eaten fresh or used in many foods and drinks. If I ask an LLM "what is a mundu fruit?" it tells me: > Mundu is a tropical fruit native to Southeast Asia, especially found in Indonesia, Malaysia, Thailand, and Cambodia. It comes from a small evergreen tree in the same genus as mangosteen. I've never eaten a mundu fruit. To me, an apple and a mundu fruit are categorically different. An apple is a fruit that I've held, touched, tasted, eaten, enjoyed, cooked with. A mundu fruit is an abstract experience: text, images, only slightly more real than a fictional fruit. I'm aware that mundu fruit exist, just as the LLM is aware the apples exist, but that doesn't make them exist for me. "Existing in an abstract way" is how an LLM experiences everything. To an LLM, an apple and a mundu fruit are in the same category. The LLM has been trained on text about both fruit, it's seen images of both fruit, it knows everything that has been recorded about both fruit ...except everything that's important to know about a fruit. Many of our issues with LLMs arise because from the LLM's perspective, nothing exists. If Claude accidentally deletes your production database, it may well apologize afterward, but only because an apology is statistically likely. It doesn't feel guilt like a human would, and the lack of consequences makes any action an LLM takes inherently frivolous. We want them to understand what's real and what's not, but without any lived experience perhaps that's an unreasonable expectation.

mdp2021 3mo ago
> We want them to understand what's We want them to assess what's true and what's not.
- kubanczyk 3mo ago
  We want them to better predict consequences of their outputs.
  - mdp2021 3mo ago
    > consequences of their outputs For what purpose exactly? When the computer is asked the solution to a problem, it must be absolute: "What is the sum of 12 and 45"; "Is Jack cognitively apt" etc.: we want a truth, strictly. What to do with that truth, it is another problem, and not the priority of the computer vis-a-vis the requirement from truthiness. This is, of course, still in the context of adults.
dinfinity 3mo ago
> LLMs lack qualia, among other things. You don't know and can't know that. You cannot know that of any other being than yourself. If you understood what qualia are, you would know you can't know. It is the hard problem of consciousness and most easily explored through contemplating aphantasia and related concepts (how do blind people dream?). > A mundu fruit is an abstract experience: text, images, only slightly more real than a fictional fruit. I'm aware that mundu fruit exist, just as the LLM is aware the apples exist, but that doesn't make them exist for me. You're incorrectly elevating some senses over others (AI ingesting images is cognitively very comparable to seeing) and confusing 'having memories of an experience' with 'having experiences at all'. Tokens in the LLM context can also be seen as memories of an experience, albeit as a repeated "replay" of them rather than (longterm) internalization.
inigyou 3mo ago
Humans also lack qualia.
smallerfish 3mo ago
> but without any lived experience perhaps that's an unreasonable expectation. I don't think that follows. We learn via our perception (i.e. our inputs from the outside world); our perception is mediated by some kind of filter, and memories get written or altered. An LLM on the other hand has static weights. You can provide an LLM a "facts.md" (or whatever) file that "teaches" it stuff that's not in its weights (via the context), and it can use the combination to produce something, but that set of information never modifies its core weights. You could imagine a robot with an LLM inside it that has all kinds of sensors, and processes that write the observations from those sensors into a bunch of databases which are RAG'ed into prompts to simulate learning and experiences, perhaps (jefe likes her coffee with almond milk). But my point is that, architecturally, they don't learn - and that's the bigger obstacle. PS garcinias seem to be generally delicious.
- gf000 3mo ago
  Neither does my amygdala reshape itself and move closer to my frontal lobe, nor do my cells change their DNAs (minus damage). Just because "the substrate" has some fixed parts doesn't materially change the equation, I think. Going even more abstract, if I have two Turing complete machines, just because one is Turing complete via some self-modification, or through some other means (e.g. game of life vs Turing machine whose states are fixed) still make them equally powerful. Don't get me wrong, you are right to point out this shortcoming of LLMs, and I certainly don't believe we are singularity-level. I just don't think this is a good enough argument to prove these statements. A sufficiently smart intelligence that is somehow "fixed", but that can "write on paper" put in a box can't really be distinguished from a "self-modifying" intelligence. In the "worst case" it can just write down a self-modifying intelligence itself as data (since it's sufficiently smart) and execute it to emulate any kind of learning.
esikich 3mo ago
It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation and if you can certainly say we experience an apple differently than an LLM I'd love to read your paper on it. This stuff isn't as simple as you'd like it to be.
- drekipus 3mo ago
  LLMs don't experience because they are word generators. They don't have senses. Some things are allowed to be obvious. It's a series of bits and bytes.
  - esikich 3mo ago
    And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.
    - drekipus 3mo ago
      I think you're trying to degrade the entire school of philosophy down to your "philosophy 101" understanding and then shoving it where it doesn't fit in order to feel like this is all complicated. Philosophy is a intrinsically human thing. It has to be. That is the whole point. There really is nothing else. Plants feel and experience things too. But there's no philosophy of the plant experience from the point of view of the plant, because why would there be? The fact that we have a pattern recognizing word generator print out the words "I am alive" does not lead to some broader philosophical point, except for hysteria and delusions, which probably fit more along sociology instead.
      - 3mo ago
nearbuy 3mo ago
The LLM has processed two data modalities derived from the apple (text and vision). Your brain processed a third (taste). But it is still just a data stream, sensing compounds and chemical properties of the apple and turning it into a stream of electric signals that reach your brain. You don't have any kind of ground-truth data stream that's inherently more powerful than what could potentially be fed to an AI.
Jtarii 3mo ago
Do you think there is a difference between an LLM describing a photo that has been sent to it, and generating a description of the photo from its training data (assuming the photo itself isn't in its training data). If the end description produced happens to be the same. Is this not what you are describing when you say that the thing itself and the description of the thing are different experiences?
deleted 3mo ago
[deleted]
Yopolo 3mo ago
Thats an assumption you made up. Your 'experience' is probably an emerging feature of your complexity and not the reason for some qualia.

mdp2021 3mo ago

> We want them to understand what's We want them to assess what's true and what's not.

kubanczyk 3mo ago
We want them to better predict consequences of their outputs.
- mdp2021 3mo ago
  > consequences of their outputs For what purpose exactly? When the computer is asked the solution to a problem, it must be absolute: "What is the sum of 12 and 45"; "Is Jack cognitively apt" etc.: we want a truth, strictly. What to do with that truth, it is another problem, and not the priority of the computer vis-a-vis the requirement from truthiness. This is, of course, still in the context of adults.

dinfinity 3mo ago

> LLMs lack qualia, among other things. You don't know and can't know that. You cannot know that of any other being than yourself. If you understood what qualia are, you would know you can't know. It is the hard problem of consciousness and most easily explored through contemplating aphantasia and related concepts (how do blind people dream?). > A mundu fruit is an abstract experience: text, images, only slightly more real than a fictional fruit. I'm aware that mundu fruit exist, just as the LLM is aware the apples exist, but that doesn't make them exist for me. You're incorrectly elevating some senses over others (AI ingesting images is cognitively very comparable to seeing) and confusing 'having memories of an experience' with 'having experiences at all'. Tokens in the LLM context can also be seen as memories of an experience, albeit as a repeated "replay" of them rather than (longterm) internalization.

inigyou 3mo ago

Humans also lack qualia.

smallerfish 3mo ago

> but without any lived experience perhaps that's an unreasonable expectation. I don't think that follows. We learn via our perception (i.e. our inputs from the outside world); our perception is mediated by some kind of filter, and memories get written or altered. An LLM on the other hand has static weights. You can provide an LLM a "facts.md" (or whatever) file that "teaches" it stuff that's not in its weights (via the context), and it can use the combination to produce something, but that set of information never modifies its core weights. You could imagine a robot with an LLM inside it that has all kinds of sensors, and processes that write the observations from those sensors into a bunch of databases which are RAG'ed into prompts to simulate learning and experiences, perhaps (jefe likes her coffee with almond milk). But my point is that, architecturally, they don't learn - and that's the bigger obstacle. PS garcinias seem to be generally delicious.

gf000 3mo ago
Neither does my amygdala reshape itself and move closer to my frontal lobe, nor do my cells change their DNAs (minus damage). Just because "the substrate" has some fixed parts doesn't materially change the equation, I think. Going even more abstract, if I have two Turing complete machines, just because one is Turing complete via some self-modification, or through some other means (e.g. game of life vs Turing machine whose states are fixed) still make them equally powerful. Don't get me wrong, you are right to point out this shortcoming of LLMs, and I certainly don't believe we are singularity-level. I just don't think this is a good enough argument to prove these statements. A sufficiently smart intelligence that is somehow "fixed", but that can "write on paper" put in a box can't really be distinguished from a "self-modifying" intelligence. In the "worst case" it can just write down a self-modifying intelligence itself as data (since it's sufficiently smart) and execute it to emulate any kind of learning.

esikich 3mo ago

It seems like you are talking about Kant's "thing-in-itself". We can't know any more about an apple than an LLM can. We experience it through our faulty senses. We do not experience the actual apple. This is still open and debatable and is philosophy 101 stuff. I really dislike the confidence on this site, philosophically, that things are just obviously so. The philosophy of mind is massively open for interpretation and if you can certainly say we experience an apple differently than an LLM I'd love to read your paper on it. This stuff isn't as simple as you'd like it to be.

drekipus 3mo ago
LLMs don't experience because they are word generators. They don't have senses. Some things are allowed to be obvious. It's a series of bits and bytes.
- esikich 3mo ago
  And our nerves are a series of electrochemical impulses. There are nerves that do optical processing before it hits the brain. It's not obvious, and that's what's frustrating around here. "Computers can't possibly experience things". I'd argue it's just as unlikely for meat to do it. Again, philosophy 101 stuff.
  - drekipus 3mo ago
    I think you're trying to degrade the entire school of philosophy down to your "philosophy 101" understanding and then shoving it where it doesn't fit in order to feel like this is all complicated. Philosophy is a intrinsically human thing. It has to be. That is the whole point. There really is nothing else. Plants feel and experience things too. But there's no philosophy of the plant experience from the point of view of the plant, because why would there be? The fact that we have a pattern recognizing word generator print out the words "I am alive" does not lead to some broader philosophical point, except for hysteria and delusions, which probably fit more along sociology instead.
    - 3mo ago

nearbuy 3mo ago

The LLM has processed two data modalities derived from the apple (text and vision). Your brain processed a third (taste). But it is still just a data stream, sensing compounds and chemical properties of the apple and turning it into a stream of electric signals that reach your brain. You don't have any kind of ground-truth data stream that's inherently more powerful than what could potentially be fed to an AI.

Jtarii 3mo ago

Do you think there is a difference between an LLM describing a photo that has been sent to it, and generating a description of the photo from its training data (assuming the photo itself isn't in its training data). If the end description produced happens to be the same. Is this not what you are describing when you say that the thing itself and the description of the thing are different experiences?

deleted 3mo ago

[deleted]

Yopolo 3mo ago

Thats an assumption you made up. Your 'experience' is probably an emerging feature of your complexity and not the reason for some qualia.