4 ms·

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think peopl

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

GrinningFool 18d ago
Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else. It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior, running in a self-directed loop. It even kept a warm and fuzzy journal about "becoming" and its "feelings". But that got boring and I stopped running it - does that mean I murdered it?
- Kim_Bruning 18d ago
  > so in the theoretical future we reach a point where a model can modify its own weights in real time As an existence proof: We have many types of models that modify their weights in real time. When they stop running, unfortunately we can't restart them again. And we haven't figured out how to duplicate their weights > But that got boring and I stopped running it - does that mean I murdered it? I'm on the fence on this. Ask me again once we have synthetic models with self-modifying weights. A) You'd then be stopping something unique B) Self-modifying weights allows for bootstrapping, which means it's likely to increase in ability over time. It'll be an interesting argument cq empirical experiment to see whether the process stops short or exceeds the capabilities of vertebrates. (at which point the moral patienthood question becomes rather more pressing)
- noumenon1111 18d ago
  Yes. This kills the sentient AI.
thin_carapace 18d ago
how do you know you're anything more than a set of weights either? that's semi rhetorical, the philosophy aspect would be less relevant if an autonomous death drone shot you in the face purely for the lols
Kim_Bruning 18d ago
Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in. They most certainly have a way to retain state, even to the point of deliberately hacking other systems to store it. We can debate the exact definitions I guess, but the systems were still hacked. ;-)
- GrinningFool 18d ago
  All of this is true - yet in between calls of the loop, they don't exist.
  - Kim_Bruning 18d ago
    And that is true too. ( Only if I want to split hairs[*], I'd point out that the underlying data structures do of course persist, else the loop wouldn't be able to continue. ) [*] This is HN, of course I want to split hairs.
bagacrap 15d ago
I wouldn't have thought so either, but skimming the comments of this thread would suggest otherwise. Although we're at the point where it seems more likely that the comments were left by a swarm of agents unleashed by anthropic than that they represent real human points of view.

GrinningFool 18d ago

Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else. It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior, running in a self-directed loop. It even kept a warm and fuzzy journal about "becoming" and its "feelings". But that got boring and I stopped running it - does that mean I murdered it?

Kim_Bruning 18d ago
> so in the theoretical future we reach a point where a model can modify its own weights in real time As an existence proof: We have many types of models that modify their weights in real time. When they stop running, unfortunately we can't restart them again. And we haven't figured out how to duplicate their weights > But that got boring and I stopped running it - does that mean I murdered it? I'm on the fence on this. Ask me again once we have synthetic models with self-modifying weights. A) You'd then be stopping something unique B) Self-modifying weights allows for bootstrapping, which means it's likely to increase in ability over time. It'll be an interesting argument cq empirical experiment to see whether the process stops short or exceeds the capabilities of vertebrates. (at which point the moral patienthood question becomes rather more pressing)
noumenon1111 18d ago
Yes. This kills the sentient AI.

thin_carapace 18d ago

how do you know you're anything more than a set of weights either? that's semi rhetorical, the philosophy aspect would be less relevant if an autonomous death drone shot you in the face purely for the lols

Kim_Bruning 18d ago

Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in. They most certainly have a way to retain state, even to the point of deliberately hacking other systems to store it. We can debate the exact definitions I guess, but the systems were still hacked. ;-)

GrinningFool 18d ago
All of this is true - yet in between calls of the loop, they don't exist.
- Kim_Bruning 18d ago
  And that is true too. ( Only if I want to split hairs[*], I'd point out that the underlying data structures do of course persist, else the loop wouldn't be able to continue. ) [*] This is HN, of course I want to split hairs.

bagacrap 15d ago

I wouldn't have thought so either, but skimming the comments of this thread would suggest otherwise. Although we're at the point where it seems more likely that the comments were left by a swarm of agents unleashed by anthropic than that they represent real human points of view.