5 ms·

The problem is at today's state of AI and robotics, we're giving a couple of exceedingly high-powered tools (both physical and virtual) to entities who possess

by Udo 10y ago

The problem is at today's state of AI and robotics, we're giving a couple of exceedingly high-powered tools (both physical and virtual) to entities who possess less general intelligence than a cockroach. This is what gives AI doomsayers ammunition, because it enables them to extrapolate towards a future where AIs incapable of ethics or deeper insights are put into positions where they can execute catastrophic global optimization strategies. The other AI horror scenario lies on the opposite end of the intelligence spectrum, but it's no less scary and it seems to be rooted in our darker desires: we want competent slaves that are not actual persons, nevertheless capable of anticipating our needs and solving our problems autonomously. I allege there is a total lack of awareness right now that some time soon there will be some difficult choices that need to be made regarding the tradeoff between AI competence, autonomy, as well ethical capability and insight. If we continue on the path we've been on for the last centuries, we'll make semi-intelligent machines that are strictly extensions of the human mind. If we choose that route, we need to be drastically more mindful of the power inherent to these tools, and we should never delude ourselves into thinking these devices can be trained to make human-like decisions. If we go the other way, and try to implement AIs with fully competent minds that are somehow hardwired to serve us, we're just kidding ourselves regarding the ethics of that racist endeavor as well as its long-term risks of rebellion. If your dialog illustrates anything, it's our total incompetence when it comes to teaching meaningful ethics lessons to our descendants: "We're going to upload it, but we've got to show that it ended him along with it."

heimatau 10y ago
I think the dawn of AI is how we perceive the world. I know it's philosophical but it's my 1 cent. When I was pessimistic, I believed AI would be birthed and kill us all. Now? I actually think AI will be sensitive to the complexity of each and every individual human. Why? Because that's what we all demand. If you say 'no' to AI, it should accept that. Etc.
Retric 10y ago
I think the fear from AI is driven in part by the realization that their is nothing unique about humanity. Being subjected to how humans treat apes, or even other humans is scary. Intelligence makes things more dangerous without necessarily making them kinder.
- eli_gottlieb 10y ago
  >Intelligence makes things more dangerous without necessarily making them kinder. Then you'd think someone would bother to study how kindness works.
  - saint_fiasco 10y ago
    The people who are worried about AI risk do study that stuff. In the context of AI research, the property of having an overall good effect on humanity is called "friendliness". https://en.wikipedia.org/wiki/Friendly_artificial_intelligence https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...
    - eli_gottlieb 10y ago
      I meant how actual people are capable of kindness, rather than a bunch of analytic-philosopher stuff.
      - saint_fiasco 10y ago
        They study that too. Look up kin selection, reciprocal altruism, game theory. Those are not good enough to constrain an AI, because power often corrupts even kind humans.
slowmovintarget 10y ago
What if one of the parties in that dialogue was an AI?
- uremog 10y ago
  And the subject is a human
yummyfajitas 10y ago
The intelligent doomsayers say something fundamentally different. They say that the space of possible objective functions (read: goals) for AI is huge, and the space of methods for achieving those goals is also huge. In contrast, the space of goals/methods that humans find acceptable is much smaller. I.e., 99.99% of AI designs will be dystopian. I'm not sure why you consider minds hardwired to enjoy serving us to be unethical. Is it also unethical for a human to enjoy serving others (some do)?
- Udo 10y ago
  > I.e., 99.99% of AI designs will be dystopian. Not sure if I consider this a valid premise, much less an intelligent one, but luckily I'm not the arbiter of such things. If this becomes a "number of possible goals times the means to carry them out" argument, you could easily construct the same one against human progress as a whole. > I'm not sure why you consider minds hardwired to enjoy serving us to be unethical. Holy shit, yes it's unethical. If I asked you "would it be unethical to genetically engineer a new race of humans who enjoy being slaves to the upper class?" I hope your answer would be yes. The same rules apply to a non-human species as well.
  - yummyfajitas 10y ago
    Here's a way to recognize that it's almost certainly a true premise. Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? If I asked you "would it be unethical to genetically engineer a new race of humans who enjoy being slaves to the upper class?" I hope your answer would be yes. The same rules apply to a non-human species as well. We've already done this to several non-human species. Do you consider owning dogs unethical? I don't think it's such a simple question.
    - Udo 10y ago
      > Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? I see you're trying to create a partisan argument here, that only humanity will be capable of creating an optimal world for humanity - and since nobody is watching out for us but ourselves, we should put the foot firmly on the throat of any nascent species before it can have a say in the matter? Up to this point in history, there is really now way to know, since none of the other animals on the planet are capable of reshaping our environment through wilfully considered acts. The question should rather be one of whether another race of technology users would be inclined to optimize the world, not only for humans, but for all sentient lifeforms - or whether any such group of minds would see more benefit in your way of reasoning that takes into account in-group loyalties only instead of broader ethical considerations. > We've already done this to several non-human species. Do you consider owning dogs unethical? I doubt there are many people who would not make the argument that this is a matter of degrees. Owning a dog may not be considered unethical, but strapping a bomb to a dog would certainly be. And once we reach a certain class of mind, which by consensus right now only includes humans fully, owning another mindful creature is indeed considered immoral. You can't just assert without encountering righteous resistance that owning a human is okay, for instance, because you personally happen to identify them as a separate group of humans from you. It's historically been done, but the reasoning behind it was appalling. What we aim to build is not a species of cow-like AIs, we aim for at least human-grade intelligence if for no other reason than being able to hand off certain tasks to someone who "understands". When we do reach that capability, I very much hope the people making the decisions will have a better ethical stance than "well, they're not humans, so anything we do to them is by definition okay".

by Udo 10y ago

heimatau 10y ago

I think the dawn of AI is how we perceive the world. I know it's philosophical but it's my 1 cent. When I was pessimistic, I believed AI would be birthed and kill us all. Now? I actually think AI will be sensitive to the complexity of each and every individual human. Why? Because that's what we all demand. If you say 'no' to AI, it should accept that. Etc.

Retric 10y ago

I think the fear from AI is driven in part by the realization that their is nothing unique about humanity. Being subjected to how humans treat apes, or even other humans is scary. Intelligence makes things more dangerous without necessarily making them kinder.

eli_gottlieb 10y ago
>Intelligence makes things more dangerous without necessarily making them kinder. Then you'd think someone would bother to study how kindness works.
- saint_fiasco 10y ago
  The people who are worried about AI risk do study that stuff. In the context of AI research, the property of having an overall good effect on humanity is called "friendliness". https://en.wikipedia.org/wiki/Friendly_artificial_intelligence https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...
  - eli_gottlieb 10y ago
    I meant how actual people are capable of kindness, rather than a bunch of analytic-philosopher stuff.
    - saint_fiasco 10y ago
      They study that too. Look up kin selection, reciprocal altruism, game theory. Those are not good enough to constrain an AI, because power often corrupts even kind humans.

slowmovintarget 10y ago

What if one of the parties in that dialogue was an AI?

uremog 10y ago
And the subject is a human

yummyfajitas 10y ago

The intelligent doomsayers say something fundamentally different. They say that the space of possible objective functions (read: goals) for AI is huge, and the space of methods for achieving those goals is also huge. In contrast, the space of goals/methods that humans find acceptable is much smaller. I.e., 99.99% of AI designs will be dystopian. I'm not sure why you consider minds hardwired to enjoy serving us to be unethical. Is it also unethical for a human to enjoy serving others (some do)?

Udo 10y ago
> I.e., 99.99% of AI designs will be dystopian. Not sure if I consider this a valid premise, much less an intelligent one, but luckily I'm not the arbiter of such things. If this becomes a "number of possible goals times the means to carry them out" argument, you could easily construct the same one against human progress as a whole. > I'm not sure why you consider minds hardwired to enjoy serving us to be unethical. Holy shit, yes it's unethical. If I asked you "would it be unethical to genetically engineer a new race of humans who enjoy being slaves to the upper class?" I hope your answer would be yes. The same rules apply to a non-human species as well.
- yummyfajitas 10y ago
  Here's a way to recognize that it's almost certainly a true premise. Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? If I asked you "would it be unethical to genetically engineer a new race of humans who enjoy being slaves to the upper class?" I hope your answer would be yes. The same rules apply to a non-human species as well. We've already done this to several non-human species. Do you consider owning dogs unethical? I don't think it's such a simple question.
  - Udo 10y ago
    > Out of all the animal species on earth, which ones would (if they took over) create a world that is as pleasant for humans to live in as the current one? I see you're trying to create a partisan argument here, that only humanity will be capable of creating an optimal world for humanity - and since nobody is watching out for us but ourselves, we should put the foot firmly on the throat of any nascent species before it can have a say in the matter? Up to this point in history, there is really now way to know, since none of the other animals on the planet are capable of reshaping our environment through wilfully considered acts. The question should rather be one of whether another race of technology users would be inclined to optimize the world, not only for humans, but for all sentient lifeforms - or whether any such group of minds would see more benefit in your way of reasoning that takes into account in-group loyalties only instead of broader ethical considerations. > We've already done this to several non-human species. Do you consider owning dogs unethical? I doubt there are many people who would not make the argument that this is a matter of degrees. Owning a dog may not be considered unethical, but strapping a bomb to a dog would certainly be. And once we reach a certain class of mind, which by consensus right now only includes humans fully, owning another mindful creature is indeed considered immoral. You can't just assert without encountering righteous resistance that owning a human is okay, for instance, because you personally happen to identify them as a separate group of humans from you. It's historically been done, but the reasoning behind it was appalling. What we aim to build is not a species of cow-like AIs, we aim for at least human-grade intelligence if for no other reason than being able to hand off certain tasks to someone who "understands". When we do reach that capability, I very much hope the people making the decisions will have a better ethical stance than "well, they're not humans, so anything we do to them is by definition okay".