5 ms·

I think before we get to self-referentiality, we have a bigger problem to solve, and that's one that I did not expect: namely, at which point do we stop saying

• amelius 1mo ago
  It's not in big-money's interest to call them conscious because that would mean they get rights too.
• euroderf 1mo ago
  Consciousness requires qualia. My 0,02€.
  • tsukikage 1mo ago
    We don't understand what those are or how they come about, so how does that help us objectively reason about whether a particular black box does or does not have them, especially when it behaves as though it does, beyond arbitrary I-know-it-when-I-see-it "things I don't like don't have the property" style declarations?
  • DonaldFisk 1mo ago
    That's the definition that matters. And as it's impossible to describe in words what it's like to experience qualia, and LLMs are trained on words, they cannot know what it's like. Even if images and sounds are included in their input training data, those are just arrays or sequences of numbers, not the experience of seeing and hearing.
    • jstanley 1mo ago
      Their training data isn't qualia itself, but neither is your training data. Qualia are something you get on the inside in response to inputs. I don't see why LLMs can't have them. I'm not saying they do, but I don't think your argument proves that they don't.
      • DonaldFisk 1mo ago
        My training data on qualia is actually seeing, hearing, and otherwise sensing things, not just reading about them.
        • fc417fc802 1mo ago
          Ah yes, because you used the magical pixie dust that makes experiences "real" therefore you're actually seeing and hearing while a multimodal LLM doesn't really experience what it sees and hears "for real". We can be certain of this because, despite not being able to quantify or even rigorously define the phenomenon we intentionally deprive the LLMs of access to said magical pixie dust.
          • t_von_doom 1mo ago
            I agree. To further ask, why is your training data objectively real and qualifies you for consciousness? Is a deaf or blind person not conscious because they don't share your training data? If a deaf person reads about sounds, their experience of them is moot? If an alien with a greater array of senses then us exists, are we therefore not conscious?
            • DonaldFisk 1mo ago
              People aren't just consciousness, they're conscious of things. Of course a deaf or blind person is conscious, even if they've been deaf or blind since birth. But they aren't conscious of hearing or seeing things, and if you ask them they'll tell you that. I'm not conscious of magnetic fields, or ultraviolet light, but other animals are, and no amount of study will overcome that deficit.
  • StevenWaterman 1mo ago
    That doesn't really solve the problem. We can't conclusively say the models don't have qualia. Hell we don't know if a perfectly accurate atom-for-atom simulation of a human brain, would produce qualia.
  • jwr 28d ago
    I would (tentatively) agree — but that doesn't invalidate my argument: qualia are not objectively measurable, so you still can't tell if a machine is conscious.
• _hao 1mo ago
  Scott Aaronson mentions Penrose's "The Emperor's New Mind", but I feel "Shadows of the Mind" is putting forward a much clearer view of Penrose's thesis. At the current stage of my life I'm quite comfortably in Camp C. "Intelligence" and "consciousness" are not algorithmic.[1] A lot of materialists are in Camp A. For some even today, LLM's are AGI. Unfortunately, the terminology is quite clearly not adequate. There's many a people that have very different ideas and feelings on what's what. The question on whether you can have "intelligence" without "consciousness" or "self-reference" are just on top of those. How can we define if a machine is "conscious" if we can't agree on what consciousness is? Panpsychists solve that problem by going the other way and saying that everything is conscious. Overall these are very interesting topics that we can all ponder. The realities at the current time are, however, that LLM's are indeed useful. And the harnessing tooling that is being employed those days solve a lot of actual usage problems. Whether they solve other fundamental issues remains to be seem IMO. [1]. I realize a lot of people probably haven't read the book, but you can ask your garden variety LLM to give you a quick rundown on the four camps that Sir Roger is using :)
  • yeellow 1mo ago
    Do you treat LLM as algorithmic? I mean, the process of computing tokens is algorithmic, also the process of training, but does it mean that LLM itself follows an algorithm, in a practical sense? If yes, how is it different to a biological system constrained by physics?
    • t_von_doom 1mo ago
      I would be interested in the answer to this question too. The visual cortex is certainly algorithmic in that light data enters, various cells and synapses react and fire and that information is ingested into something 'conscious'. Where does this then exist, and would it even be able to exist if we removed the biological algorithms at play?
    • _hao 1mo ago
      A short disclaimer before answering. Right now I'm doing a Mathematics & Physics undergrad degree and have interest in philosophy. I'm not a specialist in any of those fields. I have plans to tackle the actual math around LLM's a bit further, but I'm not there yet. To question 1 based on my current understanding (and let's say "beliefs") is that - yes, LLM's are following a probabilistic algorithm to give me the output. The same prompt won't yield the same result, but the same prompt will yield results that are quite adjacent to each other in meaning because of the training. There's randomness in the sense that you can't say which next token will be put with certainty, BUT you can be certain it will be a token that exists in the pool of tokens available. Hopefully that makes sense? For question 2. Both us and LLMs are constrained by physics. The main difference for me is the fact that at every point of our existence we're constantly changing. Our internal state and bodies are changing based on all the sensory input we get from our environment and the processes that lie underneath. Another commenter in this thread talked about qualia. That's certainly part of it as well (but again we have a definition problem). That's not to say that I think that a hypothetical future machine cannot be devised to handle a lot of these, but the level of sophistication in the biological world is such that I don't think that's realistic. Let's say that even then, that happens, would that really prove that humans work the same way as the machine?
• odyssey7 1mo ago
  It “throws” an error when “it tries” “to talk” to the “server.” We’ve been speaking as if the computers were conscious for a long time already.
• lanyard-textile 1mo ago
  Our definition of consciousness requires stakes. We associate consciousness with precious life: Life that comes with rights, needs, wants. Life has rights because it naturally demands to be continued. If it makes no difference whether it is continued, it needs no rights, and we do not consider it conscious. Life has needs because it cannot survive without. If it has no needs, it has no death. No death, no life. Life has wants because it is harder without. If it has no wants, it has no emotion: It cannot feel pain, otherwise it would want no pain. It cannot feel happiness, otherwise it would want more happiness. We don't want to build consciousness! :) Deliberately designing pain and artificial needs is a cruel exercise. We can build that, but we see no need to. The real question at the table is, are these minds? And can minds exist without consciousness? Unequivocally, yes.
• nextaccountic 1mo ago
  I think this is exactly the same problem to decide whether any human is conscious, just because they say they are conscious. Humans can act as if they are conscious too, they can even lie to themselves, their whole life, and never discover the truth about their own mind