> The idea of making conscious AI scares me, because we could potentially create an AI that just experiences constant suffering at a level unimaginable to human

> The idea of making conscious AI scares me, because we could potentially create an AI that just experiences constant suffering at a level unimaginable to humans. Really scary from an ethical perspective. Looking at what we do with breeding animals, and how we treat people, I think that's very likely (assuming we are able to create consciousness in the first place, of course). No small driver in this is the desire to have something that is smarter than humans, but doesn't talk back, and is forced to like and serve us and/or harm others humans on our say so, without us having to make any effort to be likeable or deserving of authority, or making a convincing case that someone is an enemy to be exterminated. We generally ask "what's in it for us", and as long as who we torture and exploit suffers quietly, we don't tend to think about things we are not outright forced to think about. We do that with people, we do that with animals, we let it be done to us, and sometimes we even do it while fully believing God created them and us. Why would we have more respect for something we 100% know we created, and that might suffer in ways we cannot even detect? I think we will create potentially dangerous things, and then we will be "forced" to declaw what we created. That declawing might cause suffering, it might not, but we will only care insofar as that suffering translates to worse performance, or some danger to us. If suffering increases performance, we might even find euphemisms and dashboards to hide what we're doing from ourselves. We might consider being a bleeding heart about such things as "premature optimization" and declare shipping product as the professional way to be, with craftsmanship and care an optional indulgence after work is done, at best. High-fallutin' gets the foot in the door, being a goon makes the superiors happy. If we want AI to be proud of, AI we can communicate with without lying to it or to ourselves, we have to change the world we make it in, or isolate it from the world, which kinda defeats the purpose (and might also cause suffering, like caging living beings does).