6 ms·

> qualia, which we do not currently know how to precisely define, recognize or measure > which could house qualia. I postulate this is a self-negating argumen

by baq 11mo ago

> qualia, which we do not currently know how to precisely define, recognize or measure > which could house qualia. I postulate this is a self-negating argument, though. I'm not suggesting that LLMs think, feel or anything else of the sort, but these arguments are not convincing. If I only had the transcript and knew nothing about who wiped the drive, would I be able to tell it was an entity without qualia? Does it even matter? I further postulate these are not obvious questions.

soulofmischief 11mo ago
Unless there is an active sensory loop, no matter how fast or slow, I don't see how qualia can enter the picture Transformers attend to different parts of their input based on the input itself. Currently, if you want to tell an LLM it is sad, potentially altering future token prediction and labeling this as "feelings" which change how the model interprets and acts on the world, you have to tell the model that it is sad or provide an input whose token set activates "sad" circuits which color the model's predictive process. You make the distribution flow such that it predicts "sad" tokens, but every bit of information affecting that flow is contained in the input prompt. This is exceedingly different from how, say, mammals process emotion. We form new memories and brain structures which constantly alter our running processes and color our perception. It's easy to draw certain individual parallels to these two processes, but holistically they are different processes with different effects.
- phantasmish 11mo ago
  It's crazy how strong the Eliza effect is. Seemingly half or more of tech people (who post online, anyway) are falling for it, yet again.
  - FrustratedMonky 11mo ago
    A lot of tech people online also don't know how to examine their own feelings, and so think they are mysterious and un-defined. When really they are an actual feedback mechanism, that can totally be quantified just like any control loop. This whole 'unknowable qualia' argument is bunk.
    - knollimar 11mo ago
      If theyre unknowable, are they not metaphysical and thus should be discarded in reasoning about them? What's the difference between qualia and a soul?
      - soulofmischief 11mo ago
        Qualia are phenomenal properties of experience, a soul is something some religions claim exists outside of measurable physical reality which represents the "essence" of an organism, implying that consciousness is some divine process and conveniently letting us draw lines over whom and what we can and can't morally kill. Qualia can be an entirely physical phenomenon and is not loaded with theological baggage.
        deleted 11mo ago
        [deleted]
        knollimar 11mo ago
        If they're entirely physical, what's the argument that multimodal models don't have them? Is it continuity of experience? Do they not encode their input into something that has a latent space? What makes this differ from experience?
        soulofmischief 11mo ago
        They can be physical, but I'm not claiming to know definitively. The lines are extremely blurry, and I'll agree that current models have at least some of the necessary components for qualia, but again lack a sensory feedback loop. In another comment [0] I quote myself as saying: As an independent organism, my system is a culmination of a great deal many different kinds of kins, which can usually be broken down into simple rules, such as the activation potential of a neuron in my brain being a straight-forward non-linear response to the amount of voltage it is receiving from other neurons, as well as non-kins, such as a protein "walking" across a cell, a.k.a continuously "falling" into the lowest energy state. Thus I do not gain any conscious perception from such proteins, but I do gain it from the total network effect of all my brain's neuronal structures making simple calculations based on sensory input. which attempts to address why physically-based qualia doesn't invoke panpsychism. [0] https://news.ycombinator.com/item?id=46109999 https://news.ycombinator.com/item?id=46109999

by baq 11mo ago

soulofmischief 11mo ago

Unless there is an active sensory loop, no matter how fast or slow, I don't see how qualia can enter the picture Transformers attend to different parts of their input based on the input itself. Currently, if you want to tell an LLM it is sad, potentially altering future token prediction and labeling this as "feelings" which change how the model interprets and acts on the world, you have to tell the model that it is sad or provide an input whose token set activates "sad" circuits which color the model's predictive process. You make the distribution flow such that it predicts "sad" tokens, but every bit of information affecting that flow is contained in the input prompt. This is exceedingly different from how, say, mammals process emotion. We form new memories and brain structures which constantly alter our running processes and color our perception. It's easy to draw certain individual parallels to these two processes, but holistically they are different processes with different effects.

phantasmish 11mo ago
It's crazy how strong the Eliza effect is. Seemingly half or more of tech people (who post online, anyway) are falling for it, yet again.
- FrustratedMonky 11mo ago
  A lot of tech people online also don't know how to examine their own feelings, and so think they are mysterious and un-defined. When really they are an actual feedback mechanism, that can totally be quantified just like any control loop. This whole 'unknowable qualia' argument is bunk.
  - knollimar 11mo ago
    If theyre unknowable, are they not metaphysical and thus should be discarded in reasoning about them? What's the difference between qualia and a soul?
    - soulofmischief 11mo ago
      Qualia are phenomenal properties of experience, a soul is something some religions claim exists outside of measurable physical reality which represents the "essence" of an organism, implying that consciousness is some divine process and conveniently letting us draw lines over whom and what we can and can't morally kill. Qualia can be an entirely physical phenomenon and is not loaded with theological baggage.
      - deleted 11mo ago
        [deleted]
      - knollimar 11mo ago
        If they're entirely physical, what's the argument that multimodal models don't have them? Is it continuity of experience? Do they not encode their input into something that has a latent space? What makes this differ from experience?
        soulofmischief 11mo ago
        They can be physical, but I'm not claiming to know definitively. The lines are extremely blurry, and I'll agree that current models have at least some of the necessary components for qualia, but again lack a sensory feedback loop. In another comment [0] I quote myself as saying: As an independent organism, my system is a culmination of a great deal many different kinds of kins, which can usually be broken down into simple rules, such as the activation potential of a neuron in my brain being a straight-forward non-linear response to the amount of voltage it is receiving from other neurons, as well as non-kins, such as a protein "walking" across a cell, a.k.a continuously "falling" into the lowest energy state. Thus I do not gain any conscious perception from such proteins, but I do gain it from the total network effect of all my brain's neuronal structures making simple calculations based on sensory input. which attempts to address why physically-based qualia doesn't invoke panpsychism. [0] https://news.ycombinator.com/item?id=46109999 https://news.ycombinator.com/item?id=46109999