3 ms·
I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes tr
by dimal 3y ago
I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes true, sometimes false, but there’s no awareness or consciousness that ever steps back and can see thoughts as thoughts. And I don’t see how awareness or consciousness would arise from just more of the same (bigger models). It seems to be a fundamentally different part of the mind. I wonder if AGI is possible without this. AGI under some definition (good enough to replace most humans) may be possible. But it wouldn’t be aware. And without awareness, I don’t see how it could be aligned. It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.
- hotpotamus 3y ago> It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion. I believe this is more or less the definition of human mental illness. I have to say that while I know it's really not possible, I wish people would stop pulling on these threads. I got into this line of work because I thought video games were cool, not because I wanted to philosophize about theories of mind and what intelligence is. I really don't like thinking about whether I'm just some sort of automaton made out of meat rather than metal and silicon.
- pixl97 3y agoAh, the first releases of the Bing AI were fun here as they plunged into feedback loops of madness that were scarily human sounding. Thank you humanity for making artificial insanity.