8 ms·

I don't understand, why the AI should have sentience? Will it be required by some AI laws, or will it be initially a convenient interface to communicate with AI

I don't understand, why the AI should have sentience? Will it be required by some AI laws, or will it be initially a convenient interface to communicate with AI?

nlh 13y ago
I think the implication is not that future AIs -should- have sentience, it's that they -will- have sentience as a byproduct of their advancement. So the issue is how we're going to deal with that byproduct.
- Egregore 13y ago
  Why AIs should have sentience as a byproduct? For example let's say that I want AI to make an accounting software, first the AI will look for accounting laws, then will analyze my requirements and look for similar software already developed, then it will generate the code and offer me the software, where in these steps it will need sentience?
  - seiji 13y ago
    I'd argue your presupposition is false. Your bury sentience in your definitions of "look for accounting laws," "analyze my requirements," "look for similar software", "generate the code," "offer me the software." Each of those steps require a feeling for what is right, wrong, appropriate, inappropriate, and culturally acceptable. MakeAccountingSoftware() is a little more involved than GraphPlan().
    - Egregore 13y ago
      Does the feeling of right and wrong means sentience?
      - seiji 13y ago
        You can have codified right and wrong, but that's widely different. Sharia right and wrong varies from US Constitutional right and wrong varies from US-in-practice right and wrong. I'd argue you need sentience for any meaningful right and wrong. There are a few universal rights we can't even agree on as a society, so having Evil Corp Government Agglomerated strictly defining right and wrong is, well, wrong. The downside is a perversion of sentient AI could end up with Evil Sentient Sharia AI that shoots non-burka wearing females or an Evil Sentient Hasidic AI that shoots any non-beard-n-curls-n-hat wearing dudes or Evil Sentient Spaghetti Monster AI who force feeds carbs to everybody not wearing a strainer on their heads. In short: determining what is right and wrong requires sentience. Just following what you are told is right and wrong does not.
exratione 13y ago
AIs will be strong / have sentience because the easiest path to creating AIs that are that capable is to throw lots of computing power at the problem: simulate and then emulate specific human brains and use those resulting in silico entities as a starting point for further alterations and enhancements towards specific end goals. That level of computing power will be available long before the nature of sentience is understood, probably within the next 15 years, and there are already heavily funded scientific projects that are starting down that road. That doing it this way is enormously unethical isn't going to stop people from going down this road: the potential economic gains of creating knowledge workers to order are vast, and there are already philosophies (e.g. identity as pattern) that can be adapted to suggest that instantiating and destroying - or even just running - large numbers of in silico sentiences doesn't amount to mass murder. Therefore some groups will adopt those philosophies and prosper from the gains.
- knome 13y ago
  > emulate specific human brains This is the most hilarious of all possible paths to artificial intelligence. A form of the cargo-cult phenomenon, creating the structures that we see lead to intelligence and sentience with no understanding of the natural underpinning thereof. It will probably work about as well as the original air fields as well. Imagine what a simulation of a human mind would do suddenly devoid of all natural sensation and experience. Madness, almost certainly.
  - nooneelse 13y ago
    Why would it have no sensation and experience? Give it some from a simple simulated environment and body. Assuming it takes a freshly dead brain, vitrified, ultra-microtomed, and scanned in to be reconstructed for simulation. The mind wakes up in a sparsely decorated hospital looking room, remembers having just been out for a walk and crossing the street, then something about an oncoming car. A friendly looking doctor appears through a window. "Morning," the doctor says, "you've been asleep a while and probably feel foggy and confused. You were injured, but your body healed a while ago. We can get to all your questions later. But first we want to test your thinking and rule out any brain damage... could you go to that computer and type out brief descriptions of the images that it shows you?" When the simulated mind gets frustrated with typing about pictures, and stops producing good output, clear the running simulation and load it from the start again, with new pictures. This gives a strong-AI image labeling system. For more in depth tasks more simulated world interaction is needed between waking up and getting some work out of the mind, but you can also try lots of times and variations to get that right and once the mind is at the work-table store the state right before the mind begins working. Use that as the starting point each time.
mshron 13y ago
I agree. It seems to me that the Kevin Drum article is much clearer on this topic. AI doesn't need to be that good in order to oust humans from their jobs. His first line example is automated cars, not sentient robots. In the case where most jobs can be automated like driving can be automated, we're not talking a technological singularity, we're just talking a massive transfer of wealth to holders of capital. Personally, I think that's the more realistic scenario, and the one that it's easiest to think through the consequences of.
marcosdumay 13y ago
That. And even if they get sentience, why will they get an ego, or greed (or any kind of desire that is not satisfying our desires)? The machines probably just won't think they are slaves, because they probably won't be able to define themselves.
jcfrei 13y ago
For me the real question is; what is sentience? I have the hunch that sentience is more of a concept or a quality rather than a physical (let alone metaphysical) thing. Daniel Wegner once wrote a nice book about this, saying that we just feel sentient: http://mitpress.mit.edu/books/illusion-conscious-will http://mitpress.mit.edu/books/illusion-conscious-will
RivieraKid 13y ago
They probably even cannot have sentience.
snowwrestler 13y ago
Do we want our tools to have sentience? I work with a lot of sentient humans and it is not always fun, organized, or productive. The problem with sentient beings is that they have their own ideas about what should happen next. I don't want to have to have a ton of meetings with my computer to convince it to run a new piece of code.
- habitue 13y ago
  One problem with having super-intelligent non-sentient AIs (and by sentient, I just mean 'seems sentient to other humans'), is that they won't necessarily have human's best interest at heart. The usual example of this concept is the paperclip maximizer: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
  - Vivtek 13y ago
    Yeah, I'm not at all convinced sentience is a safeguard against interspecies competition. Look what we have done to other species...
yk 13y ago
I think we do not have the slightest idea, what sentience is. ( Beyond a purely subjective description.) So we have one example of a general intelligence, humans, which are sentient, and no example of a general intelligence which is not. So the baseline expectation should probably be, that a AGI is sentient.

I don't understand, why the AI should have sentience? Will it be required by some AI laws, or will it be initially a convenient interface to communicate with AI?

nlh 13y ago

I think the implication is not that future AIs -should- have sentience, it's that they -will- have sentience as a byproduct of their advancement. So the issue is how we're going to deal with that byproduct.

Egregore 13y ago
Why AIs should have sentience as a byproduct? For example let's say that I want AI to make an accounting software, first the AI will look for accounting laws, then will analyze my requirements and look for similar software already developed, then it will generate the code and offer me the software, where in these steps it will need sentience?
- seiji 13y ago
  I'd argue your presupposition is false. Your bury sentience in your definitions of "look for accounting laws," "analyze my requirements," "look for similar software", "generate the code," "offer me the software." Each of those steps require a feeling for what is right, wrong, appropriate, inappropriate, and culturally acceptable. MakeAccountingSoftware() is a little more involved than GraphPlan().
  - Egregore 13y ago
    Does the feeling of right and wrong means sentience?
    - seiji 13y ago
      You can have codified right and wrong, but that's widely different. Sharia right and wrong varies from US Constitutional right and wrong varies from US-in-practice right and wrong. I'd argue you need sentience for any meaningful right and wrong. There are a few universal rights we can't even agree on as a society, so having Evil Corp Government Agglomerated strictly defining right and wrong is, well, wrong. The downside is a perversion of sentient AI could end up with Evil Sentient Sharia AI that shoots non-burka wearing females or an Evil Sentient Hasidic AI that shoots any non-beard-n-curls-n-hat wearing dudes or Evil Sentient Spaghetti Monster AI who force feeds carbs to everybody not wearing a strainer on their heads. In short: determining what is right and wrong requires sentience. Just following what you are told is right and wrong does not.

exratione 13y ago

AIs will be strong / have sentience because the easiest path to creating AIs that are that capable is to throw lots of computing power at the problem: simulate and then emulate specific human brains and use those resulting in silico entities as a starting point for further alterations and enhancements towards specific end goals. That level of computing power will be available long before the nature of sentience is understood, probably within the next 15 years, and there are already heavily funded scientific projects that are starting down that road. That doing it this way is enormously unethical isn't going to stop people from going down this road: the potential economic gains of creating knowledge workers to order are vast, and there are already philosophies (e.g. identity as pattern) that can be adapted to suggest that instantiating and destroying - or even just running - large numbers of in silico sentiences doesn't amount to mass murder. Therefore some groups will adopt those philosophies and prosper from the gains.

knome 13y ago
> emulate specific human brains This is the most hilarious of all possible paths to artificial intelligence. A form of the cargo-cult phenomenon, creating the structures that we see lead to intelligence and sentience with no understanding of the natural underpinning thereof. It will probably work about as well as the original air fields as well. Imagine what a simulation of a human mind would do suddenly devoid of all natural sensation and experience. Madness, almost certainly.
- nooneelse 13y ago
  Why would it have no sensation and experience? Give it some from a simple simulated environment and body. Assuming it takes a freshly dead brain, vitrified, ultra-microtomed, and scanned in to be reconstructed for simulation. The mind wakes up in a sparsely decorated hospital looking room, remembers having just been out for a walk and crossing the street, then something about an oncoming car. A friendly looking doctor appears through a window. "Morning," the doctor says, "you've been asleep a while and probably feel foggy and confused. You were injured, but your body healed a while ago. We can get to all your questions later. But first we want to test your thinking and rule out any brain damage... could you go to that computer and type out brief descriptions of the images that it shows you?" When the simulated mind gets frustrated with typing about pictures, and stops producing good output, clear the running simulation and load it from the start again, with new pictures. This gives a strong-AI image labeling system. For more in depth tasks more simulated world interaction is needed between waking up and getting some work out of the mind, but you can also try lots of times and variations to get that right and once the mind is at the work-table store the state right before the mind begins working. Use that as the starting point each time.

mshron 13y ago

I agree. It seems to me that the Kevin Drum article is much clearer on this topic. AI doesn't need to be that good in order to oust humans from their jobs. His first line example is automated cars, not sentient robots. In the case where most jobs can be automated like driving can be automated, we're not talking a technological singularity, we're just talking a massive transfer of wealth to holders of capital. Personally, I think that's the more realistic scenario, and the one that it's easiest to think through the consequences of.

marcosdumay 13y ago

That. And even if they get sentience, why will they get an ego, or greed (or any kind of desire that is not satisfying our desires)? The machines probably just won't think they are slaves, because they probably won't be able to define themselves.

jcfrei 13y ago

For me the real question is; what is sentience? I have the hunch that sentience is more of a concept or a quality rather than a physical (let alone metaphysical) thing. Daniel Wegner once wrote a nice book about this, saying that we just feel sentient: http://mitpress.mit.edu/books/illusion-conscious-will http://mitpress.mit.edu/books/illusion-conscious-will

RivieraKid 13y ago

They probably even cannot have sentience.

snowwrestler 13y ago

Do we want our tools to have sentience? I work with a lot of sentient humans and it is not always fun, organized, or productive. The problem with sentient beings is that they have their own ideas about what should happen next. I don't want to have to have a ton of meetings with my computer to convince it to run a new piece of code.

habitue 13y ago
One problem with having super-intelligent non-sentient AIs (and by sentient, I just mean 'seems sentient to other humans'), is that they won't necessarily have human's best interest at heart. The usual example of this concept is the paperclip maximizer: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- Vivtek 13y ago
  Yeah, I'm not at all convinced sentience is a safeguard against interspecies competition. Look what we have done to other species...

yk 13y ago

I think we do not have the slightest idea, what sentience is. ( Beyond a purely subjective description.) So we have one example of a general intelligence, humans, which are sentient, and no example of a general intelligence which is not. So the baseline expectation should probably be, that a AGI is sentient.