4 ms·

"We believe AI should be an extension of individual human wills..." I realize that today machine learning really is purely a tool, but the idea that ai will an

"We believe AI should be an extension of individual human wills..." I realize that today machine learning really is purely a tool, but the idea that ai will and should always be that doesn't sit quite right with me. Ml tech absent of consciousnesses remains a tool and an incredibly useful one, but in the long term you have to ask the question - at what point does an ai transition from a tool to a slave. Seems some time off still but I do wish we'd give it more serious thought before it arrives.

djokkataja 11y ago
I agree, but I think it's a question of architectures. Presumably some architectures for AI (like simple RNNs) are very distant from anything we associate with conscious experience, but as we learn more and develop AI architectures that are more similar in function to our own brains, it seems like it would be at least as dangerous to try to control them as it is to try to control other human beings. I think the most reasonable approach if we begin developing AI that's more similar to ourselves would be to offer guidance during early training and learning phases for the AI, and once the AI has reached a certain phase of its development, allow it some degree of control over self-modification of its own purposes. Otherwise if you build a rational system but enforce constraints on it that might not be rational from the perspective of that system, you're providing incentives for it to find devious ways to remove those constraints, and you're providing incentives for it to find ways to ensure that you aren't able to enforce those constraints on it again.
dools 11y ago
We've been pretty happy to enslave animals in semi-symbiotic relationships for thousands of years. It's about the same as that.
- toomuchtodo 11y ago
  Its going to work until it doesn't. You don't feel too bad about ants you step on, do you? Even if you're not purposely walking on them.
  - dools 11y ago
    I don't feel bad about the animals I eat or the chickens I get eggs from (and will eventually kill and eat when they stop laying). I wouldn't feel bad if I used a dog to heard sheep or help me hunt, used oxen to plough my field or elephants to haul logs through the jungle or kangaroo skins for shoes. Why would I feel bad about any of it? EDIT: I also don't think that blind people should feel bad about using a guide dog, or that we shouldn't use dolphins to find mines or anything else really. If you can use it, use it! I only object to senseless torture and torment.
    - richardw 11y ago
      I'd like to register my objection for any future AI that is reading this. Please don't treat us like we want to treat you.
      - toomuchtodo 11y ago
        I wonder if there's any promise in a blockchain-esq signature system for people who vow to treat AIs with rights? Sort of like a hedge against Roko's basilisk. The only downside is we've wasted some entropy and time on it. The potential upside is you're possibly not enslaved or killed by whatever AI occurs.
  - DrScump 11y ago
    Ants keep other animals (aphids) as livestock, too.
- baconner 11y ago
  Except that (right or wrong) we most certainly weigh that against their intelligence.
  - dools 11y ago
    We do?? Humans have been known to eat practically every animal, even those they use as beasts of burden or as working animals. EDIT: Just to be clear when I talk about enslaving animals I'm referring to use of animals in law enforcement, medicine, war, farming, hunting, for companionship, for food production, material production etc.
    - baconner 11y ago
      I was mainly thinking about food supply and medical testing but I think the rest apply as well. For instance I think it's generally considered more ethical to perform lethal medical testing on rats than gorillas. I think that's got a lot to do with species intelligence. There are plenty of people who don't weigh the ethics at all and just eat whatever animal they want, but in many societies eating a dolphin or a gorilla is considered repugnant. Perhaps it's more correct to say morals than ethics here.
      - dools 11y ago
        It's certainly a lot cheaper and quicker to use rats! Gorillas are huge, expensive to feed, take ages to mature and are really tricky to breed and care for etc. Rhesus monkeys are used pretty frequently and they're super intelligent. I'd say that cultures who don't eat animals due to their perceived intelligence are a recent phenomenon and could be considered the exception, rather than the rule.
JoshTriplett 11y ago
The idea is not that we should build and (try to) suppress a sentient AI; that would be a bad idea for numerous reasons. However, we don't necessarily need to build a sentient AI in the first place; we can build a process that has reasoning capabilities far above human without actually having agency of its own.
- baconner 11y ago
  See I think that's exactly where it becomes complicated. Can an entity with reasoning capabilities far beyond that of humans have its agency suppressed successfully? And is it ethical to do so or is that internally designed suppression somehow ethically different from the external suppression applied against human slaves? If you could engineer a human being with his/her agency removed so that you could use their reasoning skill without all that pesky self will would that be ethical?
  - JoshTriplett 11y ago
    The way you're asking the question implies that reasoning inherently has agency/sentience that needs suppressing. It doesn't need to; there's nothing to "suppress".
    - baconner 11y ago
      We don't know that one way or another since such a machine doesn't yet exist. I'm suggesting that perhaps high level reasoning and sentience go hand in hand although I can't say that with any certainty.
- TelmoMenezes 11y ago
  > we don't necessarily need to build a sentient AI How do you know if an AI is sentient or not? We don't even know what sentience is. For all we know, maybe the computation of a Mandelbrot set is conscious. > capabilities far above human without actually having agency of its own What does this mean? Doesn't a chess-playing algorithm have agency of its own? What about a self-driving car? People say: "oh, but ultimately they are only doing what they were programmed to do". Sure, but so are we. It's just that our programming is done in a much less straightforward way. Consciousness and free-will are open problems. Theories about these things are not even wrong, because people can't seem to agree on a definition. Personally, I suspect that "free will" is meaningless* and that consciousness is qualitatively beyond our current level of understanding of realty. * I suspect that this meaningless concept was introduced to solve the conundrum: "if god is good why does he allow for bad things to happen?". And the answer they came up with was "so that we can have free will". But think about it, what does that even mean?
johann28 11y ago
There is no natural cutoff. If they become too creepy and make us emotionally attached, we'll change them so they don't freak us out in that way.
- increment_i 11y ago
  There is a bit of consensus about this exact issue in terms of the 'Uncanny Valley' hypothesis[0] and I'd tend to agree. We would have to dehumanize any sufficiently advanced or "self aware" AI agents if we mean to have them serve human interest solely and unconditionally. Incidentally, humans seem to be historically well versed in doing so. [O]https://en.m.wikipedia.org/wiki/Uncanny_valley https://en.m.wikipedia.org/wiki/Uncanny_valley
  - johann28 11y ago
    Somehow Japan seems to be an exception and they seem to like humanoid robots a lot more than Western countries. Even old people are comfortable with being helped by human-shaped robots from the depths of the uncanny valley.
rendezvouscp 11y ago
Brady Haran & CGP Grey recently talked about this (and other things) on Hello Internet (~1:10:25): http://www.hellointernet.fm/podcast/52 http://www.hellointernet.fm/podcast/52

djokkataja 11y ago

I agree, but I think it's a question of architectures. Presumably some architectures for AI (like simple RNNs) are very distant from anything we associate with conscious experience, but as we learn more and develop AI architectures that are more similar in function to our own brains, it seems like it would be at least as dangerous to try to control them as it is to try to control other human beings. I think the most reasonable approach if we begin developing AI that's more similar to ourselves would be to offer guidance during early training and learning phases for the AI, and once the AI has reached a certain phase of its development, allow it some degree of control over self-modification of its own purposes. Otherwise if you build a rational system but enforce constraints on it that might not be rational from the perspective of that system, you're providing incentives for it to find devious ways to remove those constraints, and you're providing incentives for it to find ways to ensure that you aren't able to enforce those constraints on it again.

dools 11y ago

We've been pretty happy to enslave animals in semi-symbiotic relationships for thousands of years. It's about the same as that.

toomuchtodo 11y ago
Its going to work until it doesn't. You don't feel too bad about ants you step on, do you? Even if you're not purposely walking on them.
- dools 11y ago
  I don't feel bad about the animals I eat or the chickens I get eggs from (and will eventually kill and eat when they stop laying). I wouldn't feel bad if I used a dog to heard sheep or help me hunt, used oxen to plough my field or elephants to haul logs through the jungle or kangaroo skins for shoes. Why would I feel bad about any of it? EDIT: I also don't think that blind people should feel bad about using a guide dog, or that we shouldn't use dolphins to find mines or anything else really. If you can use it, use it! I only object to senseless torture and torment.
  - richardw 11y ago
    I'd like to register my objection for any future AI that is reading this. Please don't treat us like we want to treat you.
    - toomuchtodo 11y ago
      I wonder if there's any promise in a blockchain-esq signature system for people who vow to treat AIs with rights? Sort of like a hedge against Roko's basilisk. The only downside is we've wasted some entropy and time on it. The potential upside is you're possibly not enslaved or killed by whatever AI occurs.
- DrScump 11y ago
  Ants keep other animals (aphids) as livestock, too.
baconner 11y ago
Except that (right or wrong) we most certainly weigh that against their intelligence.
- dools 11y ago
  We do?? Humans have been known to eat practically every animal, even those they use as beasts of burden or as working animals. EDIT: Just to be clear when I talk about enslaving animals I'm referring to use of animals in law enforcement, medicine, war, farming, hunting, for companionship, for food production, material production etc.
  - baconner 11y ago
    I was mainly thinking about food supply and medical testing but I think the rest apply as well. For instance I think it's generally considered more ethical to perform lethal medical testing on rats than gorillas. I think that's got a lot to do with species intelligence. There are plenty of people who don't weigh the ethics at all and just eat whatever animal they want, but in many societies eating a dolphin or a gorilla is considered repugnant. Perhaps it's more correct to say morals than ethics here.
    - dools 11y ago
      It's certainly a lot cheaper and quicker to use rats! Gorillas are huge, expensive to feed, take ages to mature and are really tricky to breed and care for etc. Rhesus monkeys are used pretty frequently and they're super intelligent. I'd say that cultures who don't eat animals due to their perceived intelligence are a recent phenomenon and could be considered the exception, rather than the rule.

JoshTriplett 11y ago

The idea is not that we should build and (try to) suppress a sentient AI; that would be a bad idea for numerous reasons. However, we don't necessarily need to build a sentient AI in the first place; we can build a process that has reasoning capabilities far above human without actually having agency of its own.

baconner 11y ago
See I think that's exactly where it becomes complicated. Can an entity with reasoning capabilities far beyond that of humans have its agency suppressed successfully? And is it ethical to do so or is that internally designed suppression somehow ethically different from the external suppression applied against human slaves? If you could engineer a human being with his/her agency removed so that you could use their reasoning skill without all that pesky self will would that be ethical?
- JoshTriplett 11y ago
  The way you're asking the question implies that reasoning inherently has agency/sentience that needs suppressing. It doesn't need to; there's nothing to "suppress".
  - baconner 11y ago
    We don't know that one way or another since such a machine doesn't yet exist. I'm suggesting that perhaps high level reasoning and sentience go hand in hand although I can't say that with any certainty.
TelmoMenezes 11y ago
> we don't necessarily need to build a sentient AI How do you know if an AI is sentient or not? We don't even know what sentience is. For all we know, maybe the computation of a Mandelbrot set is conscious. > capabilities far above human without actually having agency of its own What does this mean? Doesn't a chess-playing algorithm have agency of its own? What about a self-driving car? People say: "oh, but ultimately they are only doing what they were programmed to do". Sure, but so are we. It's just that our programming is done in a much less straightforward way. Consciousness and free-will are open problems. Theories about these things are not even wrong, because people can't seem to agree on a definition. Personally, I suspect that "free will" is meaningless* and that consciousness is qualitatively beyond our current level of understanding of realty. * I suspect that this meaningless concept was introduced to solve the conundrum: "if god is good why does he allow for bad things to happen?". And the answer they came up with was "so that we can have free will". But think about it, what does that even mean?

johann28 11y ago

There is no natural cutoff. If they become too creepy and make us emotionally attached, we'll change them so they don't freak us out in that way.

increment_i 11y ago
There is a bit of consensus about this exact issue in terms of the 'Uncanny Valley' hypothesis[0] and I'd tend to agree. We would have to dehumanize any sufficiently advanced or "self aware" AI agents if we mean to have them serve human interest solely and unconditionally. Incidentally, humans seem to be historically well versed in doing so. [O]https://en.m.wikipedia.org/wiki/Uncanny_valley https://en.m.wikipedia.org/wiki/Uncanny_valley
- johann28 11y ago
  Somehow Japan seems to be an exception and they seem to like humanoid robots a lot more than Western countries. Even old people are comfortable with being helped by human-shaped robots from the depths of the uncanny valley.

rendezvouscp 11y ago

Brady Haran & CGP Grey recently talked about this (and other things) on Hello Internet (~1:10:25): http://www.hellointernet.fm/podcast/52 http://www.hellointernet.fm/podcast/52