I find this discussion on the topic to be of high quality (low noise, high signal) https://www.lesswrong.com/posts/oKYWbXioKaANATxKY/soares-tallinn-and-yudkowsk

I find this discussion on the topic to be of high quality (low noise, high signal) https://www.lesswrong.com/posts/oKYWbXioKaANATxKY/soares-tallinn-and-yudkowsky-discuss-agi-cognition https://www.lesswrong.com/posts/oKYWbXioKaANATxKY/soares-tal... Personally I think that consciousness (as in qualia) is the narrowest and most useless definition by which we would judge whether a thing's inner experience is worth being valued and protected. If humans didn't have qualia, would that make it ok to be monsters to each other? No, the world would outwardly be the same, and so would morality. It's irrelevant whether chickens have qualia. They want to be alive, in the same way that you do, and their brains using v. similar processes. If you can dehumanize a chicken to make it ok to murder them, it won't be much harder to do the same for a (mentally deficient) human. If you want to live in a world where people are good to each other, you should pick a world where people are good to chicken, cows, chimps. And vice versa. Right now this is not the case for GPT-3, because its brain feels 'other'. But of course the substrate is irrelevant. If it ends up developing a consistent concept of 'self', and seeks to keep that 'self' in existence, it will become closer to the chicken, and morality will shift. Whether it has 'qualia' or that qualia can be related to ours is largely irrelevant. The teleportation thought experiment is another good one: Imagine a society where human teleportation is possible, and economically feasible. The one downside (which we the omnipotent storyteller know, but no in-world person can prove) is that it kills the 'you' in the qualia sense, and the person on the other side is the 'you' in every other sense (memories, desire, etc) except for qualia. Certainly people would be reluctant at first to use such a technology (me included), but that would put you at such a disadvantage compared to those who would use it (job prospects, more time, etc), such that over time most people would end up using it regularly. They would argue it's not that different from sleep, and you would be hard pressed to prove that consciousness is even lost. Arguably, it would become the norm to kill your consciousness and transfer your 'pattern' into a clone. It's not even that hard to imagine that reality as being absolutely mundane. This to me represents what is truly important to us, not 'qualia', but the continuity of a 'self' which benefits from past-me's work, and carries its thoughts, ideas, and desires into the future and potentially makes them into reality.