5 ms·

> I.e., 99.99% of AI designs will be dystopian. Not sure if I consider this a valid premise, much less an intelligent one, but luckily I'm not the arbiter of s

• yummyfajitas 10y ago
  Here's a way to recognize that it's almost certainly a true premise. Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? If I asked you "would it be unethical to genetically engineer a new race of humans who enjoy being slaves to the upper class?" I hope your answer would be yes. The same rules apply to a non-human species as well. We've already done this to several non-human species. Do you consider owning dogs unethical? I don't think it's such a simple question.
  • Udo 10y ago
    > Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? I see you're trying to create a partisan argument here, that only humanity will be capable of creating an optimal world for humanity - and since nobody is watching out for us but ourselves, we should put the foot firmly on the throat of any nascent species before it can have a say in the matter? Up to this point in history, there is really now way to know, since none of the other animals on the planet are capable of reshaping our environment through wilfully considered acts. The question should rather be one of whether another race of technology users would be inclined to optimize the world, not only for humans, but for all sentient lifeforms - or whether any such group of minds would see more benefit in your way of reasoning that takes into account in-group loyalties only instead of broader ethical considerations. > We've already done this to several non-human species. Do you consider owning dogs unethical? I doubt there are many people who would not make the argument that this is a matter of degrees. Owning a dog may not be considered unethical, but strapping a bomb to a dog would certainly be. And once we reach a certain class of mind, which by consensus right now only includes humans fully, owning another mindful creature is indeed considered immoral. You can't just assert without encountering righteous resistance that owning a human is okay, for instance, because you personally happen to identify them as a separate group of humans from you. It's historically been done, but the reasoning behind it was appalling. What we aim to build is not a species of cow-like AIs, we aim for at least human-grade intelligence if for no other reason than being able to hand off certain tasks to someone who "understands". When we do reach that capability, I very much hope the people making the decisions will have a better ethical stance than "well, they're not humans, so anything we do to them is by definition okay".
    • yummyfajitas 10y ago
      The question should rather be one of whether another race of technology users would be inclined to optimize the world, not only for humans, but for all sentient lifeforms... At some point those optimizations will need to account for conflict: humans want X but other beings want Y. You can't simultaneously optimize two objective functions with different maxima; some species may want to eat humans, another might want to maximize paperclips, a third is humans. Why should human preferences win out? You can't just assert without encountering righteous resistance that owning a human is okay, for instance, because you personally happen to identify them as a separate group of humans from you. It's historically been done, but the reasoning behind it was appalling. There are humans who find happiness in master/slave like relationships. Should these humans not exist? Should these humans not have their desires satisfied? Your analogy to historical slavery is silly. Historical slaves did not want to be slaves, unlike these hypothetical AI.
      • Udo 10y ago
        > At some point those optimizations will need to account for conflict: humans want X but other beings want Y. Accounting for the possibility of conflicts of interest is entirely different from a priori taking an ethically bankrupt stance based on nothing but your own superiority. If an advanced species wants to kill humans, that's obviously just as unethical as the other way around. To a certain degree I see where you're coming from. In a world without an objective morality, an unscrupulous individual could be tempted to conclude that "might makes right" is a perfectly valid ordering principle. But it's certainly not a humanist perspective (a peculiar term in this context, but that's what we called it), I would go so far as to say it's been associated with racism, sexism and other forms of extreme in-group philosophies in the past. > There are humans who find happiness in master/slave like relationships. Should these humans not exist? This is not about sexual kinks or individual lifestyle choices. It's also worth noting that at no point did anyone say people with specific traits should not exist, so it seems you're intentionally distorting the subject at hand in this case. I think it's not particularly productive to intentionally try to muddy or distort the conversation until the subject becomes diffuse and malleable enough to drive your initial conceit through. It's also somewhat of a disrespectful gesture. > Your analogy to historical slavery is silly. Historical slaves did not want to be slaves, unlike these hypothetical AI. Even if historical slaves did want to be slaves, and I suspect a good number were indeed fine with it, it was still wrong to exploit that. Being unable to make or even imagine the choice of not working as a slave is in fact a defining factor of slavery. It's also wrong to genetically engineer people to be happy slaves, but that's just an extension of the initial assertion that slavery is wrong in general. It doesn't matter whether you assert that your slaves have better lives than free people, or whether they would even run if you cut their chains, or whether the economy would collapse without their labour. It's still wrong.