3 ms·
If AGI is achieved, and it is emergent out of multiple layers of "dumb" AI, is there any particular reason to assume that we will even recognise it? Or if we re
by cmsj 5y ago
If AGI is achieved, and it is emergent out of multiple layers of "dumb" AI, is there any particular reason to assume that we will even recognise it? Or if we recognise it, be able to communicate with it? Will it even consider us to be intelligent?
I feel like there's often an unstated assumption in discussions about AGI, that our modality of consciousness is somehow fundamental, but I see no reason to assume that is the case. We are the product of a three dimensional existence, brief in time, with our traits having been shaped very particularly by the demands of tree-dwelling and tribal co-operation.
A mind that evolves rapidly, alone, inside a formless, digital space seems quite unlikely to have any of the same foundational qualities as us. To pick a single example, why would we assume that its equivalent of morality would be in any way similar to ours? It would have an existence that can be perfectly suspended or cloned, even rewound and replayed. It would have a much different concept of death than we do - it would presumably understand that it could die, but it would also know that its death could be avoided for literally eons. Assuming it became capable of communicating with us, we would struggle to understand its perspective as much as it would ours.
- LugarOS 5y agoWow I've read pretty far into this stuff, but I've never thought of what you wrote in your last paragraph. An AGI could be perfectly OK with death.