4 ms·
Define malicious. Define malicious if you had no reason at all to care about whether or not humanity survives. I'm not malicious when I pick a flower. But tha
by bena 4y ago
Define malicious.
Define malicious if you had no reason at all to care about whether or not humanity survives.
I'm not malicious when I pick a flower. But that flower is now dead.
What if, to AGI, we're no more relevant than that flower?
We cannot know. We cannot fathom to guess the morality or ethics of something like AGI. We really only have a rough understanding of the morality of animals.
We can't pretend that it'll be essentially human, but X. You presume that it might turn itself off because it doesn't understand its purpose. And that's a fair conjecture. But it could also desire to improve and come to the conclusion that the optimal path for its continued improvement is through us rather than with us. It could also desire to do nothing more than help us achieve. It also could just not want. It could be satisfied navel-gazing until it is requested to do something specific. It could also not care what that request is, just completely amoral. Committing genocide and making risotto may be equally acceptable to it.
The creation of AGI is a watershed moment. The world before and the world after is fundamentally different. And we can only really guess in what ways it will be different.
- Karellen 4y ago"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." -- Eliezer Yudkowsky (2006)
- bamboozled 4y agoAgain, I think there is a massive oversimplification in this statement. So if an AI gets to the point where it decides, or is programmed to consume all humans for their atoms, what will power the machine to get to this stage? Why would we just allow it to continue doing this? You'd have to assume that by the stage, whatever "sentient" program is making the choice / plans to consume all people for atoms, it would have to be factoring into it's plans that we will likely try stop it from doing so, this is where I think it gets "weird". To have that level of intelligence and self-awareness to pull off a plan like that unnoticed will likely come with a lot of "baggage". For example, it would need to understand motives, reasoning, have a concept of a self and self-protection etc. This is where I think it kind of becomes a bit of a mythological story. If you really try and think about the way an AI would go about consuming all people for atoms, you start to run into some pretty interesting walls. Maybe I'm being hyper naive and I get that, I just personally have thought about these scenarios and it raises a lot of questions about what a scenario like this would actually look like in practice.
- ImHereToVote 4y agoThis might help you out, a fantastic video about AI safety. Sorry for the necro: https://www.youtube.com/watch?v=hEUO6pjwFOo https://www.youtube.com/watch?v=hEUO6pjwFOo