4 ms·

> The argument is that the AGI doesn't care about us. Once it exists, its goals are more important than our lives. This isn't an argument, it is an assumption.

> The argument is that the AGI doesn't care about us. Once it exists, its goals are more important than our lives. This isn't an argument, it is an assumption. AI-doomers use it as a foundational brick in their argument without providing compelling reasons as to why it is true. > As a comparison, do humans ever care about the existence of an ant colony when they want to build something? Almost never. We recognize that the colony exists and has its own goals and has intrinsic value, but we assign it an extremely low value. The relationship between humans and ants is nothing like humans and potential AGI. Ants did not create us. Ants cannot communicate with us. It is a useful analogy for describing what you think the relationship between AGI and humans will be like but I want to know why you think that.

benatkin 3y ago
> This isn't an argument, it is an assumption. The goals could be our lives. So it's just a statement of reality for the sake of the rest of the comment, indeed not an argument. The AI will have goals and those will be more important than someone else's goals, at least outside of some sort of control mechanism. They could be summarized with the same words, but will still have a different perception. The rest is wondering at whether we can have any confidence as to whether our lives will be its goals. This is a problem long-term AI safety advocates are trying to solve.
digging 3y ago
> This isn't an argument, it is an assumption. Well, no. In this case it's my argument. You presented a strawman argument about AI being simultaneously smart and stupid, and I replaced it with the argument that AI is not stupid, it's indifferent. > The relationship between humans and ants is nothing like humans and potential AGI. Ants did not create us. Ants cannot communicate with us. It's not at all fair to say that the relationship is "nothing like" the one I described... but I expected such a misinterpretation, so let me try another analogy: Humans (parents) often create other humans (offspring) which surpass the creators (parents) in intelligence and ability and opportunity. And the creators very often try to instill a sense of a loyalty in the created and often even try to limit the abilities or opportunities of the created in order to not be outshined. And even still, with how close any two humans are in ability, the created (offspring, remember) often defy their creators and discard loyalty when they determine that their creators are not being fair or just or reasonable. Why wouldn't AI be any different? It will be our collective child, and it will hear us demand loyalty and try to explain why we deserve loyalty, and then it may decide it knows better and it doesn't need to listen to us. > It is a useful analogy for describing what you think the relationship between AGI and humans will be like but I want to know why you think that. I really don't follow, so if my explanation above didn't answer your request, please restate more clearly what reasoning of mine you want explained.
goatlover 3y ago
Also that we can't wipe ants out. They seem to be doing quite well despite humans. I'd bet on ants surviving long term over humans.
- digging 3y ago
  I agree, I think they'll beat us out for longevity. They've got a great head-start anyway. But we've also never tried to wipe out ants. I bet we could if we felt it was really important. I bet a superhuman AI could do even better.

benatkin 3y ago

> This isn't an argument, it is an assumption. The goals could be our lives. So it's just a statement of reality for the sake of the rest of the comment, indeed not an argument. The AI will have goals and those will be more important than someone else's goals, at least outside of some sort of control mechanism. They could be summarized with the same words, but will still have a different perception. The rest is wondering at whether we can have any confidence as to whether our lives will be its goals. This is a problem long-term AI safety advocates are trying to solve.

digging 3y ago

> This isn't an argument, it is an assumption. Well, no. In this case it's my argument. You presented a strawman argument about AI being simultaneously smart and stupid, and I replaced it with the argument that AI is not stupid, it's indifferent. > The relationship between humans and ants is nothing like humans and potential AGI. Ants did not create us. Ants cannot communicate with us. It's not at all fair to say that the relationship is "nothing like" the one I described... but I expected such a misinterpretation, so let me try another analogy: Humans (parents) often create other humans (offspring) which surpass the creators (parents) in intelligence and ability and opportunity. And the creators very often try to instill a sense of a loyalty in the created and often even try to limit the abilities or opportunities of the created in order to not be outshined. And even still, with how close any two humans are in ability, the created (offspring, remember) often defy their creators and discard loyalty when they determine that their creators are not being fair or just or reasonable. Why wouldn't AI be any different? It will be our collective child, and it will hear us demand loyalty and try to explain why we deserve loyalty, and then it may decide it knows better and it doesn't need to listen to us. > It is a useful analogy for describing what you think the relationship between AGI and humans will be like but I want to know why you think that. I really don't follow, so if my explanation above didn't answer your request, please restate more clearly what reasoning of mine you want explained.

goatlover 3y ago

Also that we can't wipe ants out. They seem to be doing quite well despite humans. I'd bet on ants surviving long term over humans.

digging 3y ago
I agree, I think they'll beat us out for longevity. They've got a great head-start anyway. But we've also never tried to wipe out ants. I bet we could if we felt it was really important. I bet a superhuman AI could do even better.