4 ms·

It’s a static model, it does not learn after the initial training. That alone rules out any possibility for consciousness imho

obscur 4y ago
I think I heard of people with brain damage who does not keep an internal state, they feel like they continuously wake up and only remember memories from before the brain damage. That would be a kind of parallel to consider, or at least somewhat analogous. I think most of us assume that such people would have phenomenal consciousness / subjective experience / qualia
- deleted 4y ago
  [deleted]
- pantalaimon 4y ago
  I don't think that's a fair comparison. You can't put a biological system into 'read-only' mode like you can with a computer system (where this is the default since modification requires dedicated effort). A biological system self-modifies even when just associating memories. The phenomenon you describe is much more high level, we observe someone can't form new memories, but that doesn't mean their fundamental brain chemistry has changed.
woojoo666 4y ago
Then the question is whether it was conscious during training
guelo 4y ago
It should have a feedback loop from the conversations so that it could keep learning. Maybe by users rating each response.
FeepingCreature 4y ago
It could learn things via the context window. Maybe there'd be no long-term conscious state but you could keep short-term coherence well enough. I think the real problem is that it simply isn't capable of training for self-awareness and reflection because there's only very indirect evidence of it in the training set. We talk about Chains of Thought, and it's a very powerful technique, but this is all just the tiny snippets of text on the internet where people happened to narrate their thought process out loud. If it could learn on self-generated instances of using Chain of Thought to help it predict other text in its training data, this tiny fragmentary piece of consciousness would get massively reinforced.
2sk21 4y ago
Exactly - tt has no memory except the narrow window. When these models develop a hippocampus analog and can push things into long term memory, it might become interesting.

obscur 4y ago

I think I heard of people with brain damage who does not keep an internal state, they feel like they continuously wake up and only remember memories from before the brain damage. That would be a kind of parallel to consider, or at least somewhat analogous. I think most of us assume that such people would have phenomenal consciousness / subjective experience / qualia

deleted 4y ago
[deleted]
pantalaimon 4y ago
I don't think that's a fair comparison. You can't put a biological system into 'read-only' mode like you can with a computer system (where this is the default since modification requires dedicated effort). A biological system self-modifies even when just associating memories. The phenomenon you describe is much more high level, we observe someone can't form new memories, but that doesn't mean their fundamental brain chemistry has changed.

woojoo666 4y ago

Then the question is whether it was conscious during training

guelo 4y ago

It should have a feedback loop from the conversations so that it could keep learning. Maybe by users rating each response.

FeepingCreature 4y ago

It could learn things via the context window. Maybe there'd be no long-term conscious state but you could keep short-term coherence well enough. I think the real problem is that it simply isn't capable of training for self-awareness and reflection because there's only very indirect evidence of it in the training set. We talk about Chains of Thought, and it's a very powerful technique, but this is all just the tiny snippets of text on the internet where people happened to narrate their thought process out loud. If it could learn on self-generated instances of using Chain of Thought to help it predict other text in its training data, this tiny fragmentary piece of consciousness would get massively reinforced.

2sk21 4y ago

Exactly - tt has no memory except the narrow window. When these models develop a hippocampus analog and can push things into long term memory, it might become interesting.