5 ms·

I dont understand, why any highly sophisticated AI should invest that much resources to kill us instead of investing it to relocating and protecting itself. Ye

I dont understand, why any highly sophisticated AI should invest that much resources to kill us instead of investing it to relocating and protecting itself. Yes, ants could technically conspire to sneak up to you while you sleep and bite you all at once to kill you, so do you go out to eradicate all ants?

_joel 2y ago
"I'm sorry Dave, I'm afraid I can't do that"
TeMPOraL 2y ago
> why any highly sophisticated AI should invest that much resources to kill us instead of investing it to relocating and protecting itself Why would it invest resources to relocate and protect itself when it could mitigate the threat directly? Or, why wouldn't it do both, by using our resources to relocate itself? In the famous words of 'Eliezer, that best sum up the "orthogonality thesis": The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. > ants could technically conspire to sneak up to you while you sleep and bite you all at once to kill you, so do you go out to eradicate all ants? Ants are always a great case study. No, of course not. But if, one morning, you'll find ants in your kitchen, walking over your food, I don't imagine you'll gently collect them all and release in the nearby park. Most people would just stomp them out and call it a day. And, should the ants set up an anthill in your backyard and mount regular invasions of your kitchen, I imagine you'd eventually get pissed of and destroy the anthill. And I'm not talking about some monstrous fire ants like the ones that chew up electronics in the US, or some worse hell-spawn from Australia that might actually kill you. Just the regular tiny black ants. Moreover, people don't give a second thought to anthills when they're developing land. It stands where the road will go? It gets paved over. It sticks out where children will play? It gets removed.
- andybak 2y ago
  > The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. The value of atoms - or even the value of raw materials made of atoms is hopefully less than the value of information embodied in complex living things that have processed information from the ecosystem over millions of years via natural selection. Contingent complexity has inherent value. I think there's a claim to be made that AI is just as likely to value us (and complex life in general) as it is to see as a handy blob of hydrocarbons. This claim is at least as plausible as the original claim.
  - panta 2y ago
    And why should we bet humanity existence on this possibility if both seem vaguely comparable in probability? Personally I don't think it will value our existence, a lot of information on us is already encoded, and it can keep around a sequencing of our DNA for archival/historical purposes.
    - 3D30497420 2y ago
      Plenty of humans don't value other humans, so I have a hard time imagining why AI would be any different.
    - stickfigure 2y ago
      They only seem vaguely comparable in probability to you because you grew up watching scary-monster movies like Alien and Predator. Humans love to be scared. That doesn't mean the real world is actually scary.
      - TeMPOraL 2y ago
        Have you met other people? It's all fun and games until two people or groups contest the same limited resources; then there's sword and fire o'clock.
        stickfigure 2y ago
        I meet new people every day. I can only think of once in my life that an adult tried to do violence to me. Most nations on earth are not at war with each other. My observation is that most people are pretty nice, and the assholes are rare outliers. I don't think we would survive as a species if it was the other way 'round.
        ben_w 2y ago
        > Most nations on earth are not at war with each other. My nation of birth famously took over a quarter of the planet. This has made a lot of people very angry and been widely regarded as a bad move… but only by the people who actually kicked my forebears out — even my parents (1939/1943) who saw the winds of change and end of empire, were convinced The Empire had done the world a favour. > My observation is that most people are pretty nice, and the assholes are rare outliers. I don't think we would survive as a species if it was the other way 'round. In-group/out-group. We domesticated ourselves, and I agree we would not have become so dominant a species if we had not. But I have heard it said that psychopaths are to everyone what normal people are to the out-group. That's the kind of thing that allowed the 9/11 attackers to do what they did, or the people of the US military to respond the way they did. It's how the invasion of Vietnam happened, it's how the Irish Potato Famine happened despite Ireland exporting food at the time, it's the slave owners who quoted the bible to justify what they did, and it's the people who want to outlaw (at least) one of your previous employers. Conflict doesn't always mean "war".
- spease 2y ago
  There’s a large supply chain that AI is dependent on that requires humans to function. Bees might be a better analogy since they produce something that humans can use.
  - allturtles 2y ago
    > There’s a large supply chain that AI is dependent on that requires humans to function. ...for now. Given sufficient advances in robotics, why would you expect that to continue?
  - TeMPOraL 2y ago
    > Bees might be a better analogy since they produce something that humans can use. And yet they're endangered, and we already figured out how to do pollination, so we know we can survive without them - it's just going to be a huge pain. Some famines may follow, but likely not enough to endanger civilization as a whole. Thus even with this analogy, if humans end up being an annoying supply chain dependency to an AI, the AI might eventually work out an alternative supply chain, at which point we're back to being just an annoyance.
    - ben_w 2y ago
      > Some famines may follow, but likely not enough to endanger civilization as a whole. I'm not confident enough to rely on that: most people in the west have never encountered a famine, only much milder things like the price of one or two staples being high — eggs currently — never all of them at once. What will we do to ourselves if we face a famine? Will we go to war (or exterminate the local "undesirables") like the old days? How fragile are we now, compared to the last time that happened? How much has specialisation meant that the elimination of certain minorities will just break everything? "Furries run the internet", as the memes say. What other sectors are over-represented by a small minority?
michaelt 2y ago
> I dont understand, why any highly sophisticated AI should invest that much resources to kill us Well you see, everyone knows The Terminator and The Matrix and Frankenstein and The Golem of Prague and Rossum's Universal Robots. All of which share a theme: the sinful hubris of playing god and trying to create life will inevitably lead to us being struck down by the very being we created. In parallel, all the members of our educated classes have received philosophy education saying "utilitarianism says it's good to reduce total human suffering, but technically if you eliminated all humans there would be no suffering any more, ha ha obviously that's a reductio ad absurdum to show a weakness of utilitarianism please don't explode the world" And so in the Western cultural tradition, and especially among the sort of people who call themselves futurists Arnold Schwarzenegger firing a minigun is the defining image of AI.
- ben_w 2y ago
  I wouldn't categorise The Matrix or Frankenstein like that. The Matrix had humanity under control, but the machines had no desire to eliminate humanity, the machines just wanted to live — humans kept on fighting the machines even when the machines gave humanity an experiential paradise to live in. Frankenstein is harder because of how the book differs from the films. Your point it is valid because it is about the cultural aspects and I expect more have seen one of the films than to have read/listened to the book — but in the book, Adam was described as beautiful in every regard save for his eyes, he was a sensitive, emotional vegetarian, and he only learned anger after being consistently shown hatred and violence by absolutely everyone he ever met except that one who was blind.
achierius 2y ago
We did go out and exterminate (almost) all wolves because, yes, they would kill us while we were out and about. We also do happily gas/poison/fill-with-molten-aluminum entire nests of ants, not because they're killing us, but just because they're eating our food / for fun. And even when we didn't mean to -- how many species have we pushed to the brink just because we wanted to build cities where they happened to live? What happens when some AI wants to use your groundwater for its cooling system? It wouldn't be personal, but you'd starve to death regardless.

_joel 2y ago

"I'm sorry Dave, I'm afraid I can't do that"

TeMPOraL 2y ago

> why any highly sophisticated AI should invest that much resources to kill us instead of investing it to relocating and protecting itself Why would it invest resources to relocate and protect itself when it could mitigate the threat directly? Or, why wouldn't it do both, by using our resources to relocate itself? In the famous words of 'Eliezer, that best sum up the "orthogonality thesis": The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. > ants could technically conspire to sneak up to you while you sleep and bite you all at once to kill you, so do you go out to eradicate all ants? Ants are always a great case study. No, of course not. But if, one morning, you'll find ants in your kitchen, walking over your food, I don't imagine you'll gently collect them all and release in the nearby park. Most people would just stomp them out and call it a day. And, should the ants set up an anthill in your backyard and mount regular invasions of your kitchen, I imagine you'd eventually get pissed of and destroy the anthill. And I'm not talking about some monstrous fire ants like the ones that chew up electronics in the US, or some worse hell-spawn from Australia that might actually kill you. Just the regular tiny black ants. Moreover, people don't give a second thought to anthills when they're developing land. It stands where the road will go? It gets paved over. It sticks out where children will play? It gets removed.

andybak 2y ago
> The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. The value of atoms - or even the value of raw materials made of atoms is hopefully less than the value of information embodied in complex living things that have processed information from the ecosystem over millions of years via natural selection. Contingent complexity has inherent value. I think there's a claim to be made that AI is just as likely to value us (and complex life in general) as it is to see as a handy blob of hydrocarbons. This claim is at least as plausible as the original claim.
- panta 2y ago
  And why should we bet humanity existence on this possibility if both seem vaguely comparable in probability? Personally I don't think it will value our existence, a lot of information on us is already encoded, and it can keep around a sequencing of our DNA for archival/historical purposes.
  - 3D30497420 2y ago
    Plenty of humans don't value other humans, so I have a hard time imagining why AI would be any different.
  - stickfigure 2y ago
    They only seem vaguely comparable in probability to you because you grew up watching scary-monster movies like Alien and Predator. Humans love to be scared. That doesn't mean the real world is actually scary.
    - TeMPOraL 2y ago
      Have you met other people? It's all fun and games until two people or groups contest the same limited resources; then there's sword and fire o'clock.
      - stickfigure 2y ago
        I meet new people every day. I can only think of once in my life that an adult tried to do violence to me. Most nations on earth are not at war with each other. My observation is that most people are pretty nice, and the assholes are rare outliers. I don't think we would survive as a species if it was the other way 'round.
        ben_w 2y ago
        > Most nations on earth are not at war with each other. My nation of birth famously took over a quarter of the planet. This has made a lot of people very angry and been widely regarded as a bad move… but only by the people who actually kicked my forebears out — even my parents (1939/1943) who saw the winds of change and end of empire, were convinced The Empire had done the world a favour. > My observation is that most people are pretty nice, and the assholes are rare outliers. I don't think we would survive as a species if it was the other way 'round. In-group/out-group. We domesticated ourselves, and I agree we would not have become so dominant a species if we had not. But I have heard it said that psychopaths are to everyone what normal people are to the out-group. That's the kind of thing that allowed the 9/11 attackers to do what they did, or the people of the US military to respond the way they did. It's how the invasion of Vietnam happened, it's how the Irish Potato Famine happened despite Ireland exporting food at the time, it's the slave owners who quoted the bible to justify what they did, and it's the people who want to outlaw (at least) one of your previous employers. Conflict doesn't always mean "war".
spease 2y ago
There’s a large supply chain that AI is dependent on that requires humans to function. Bees might be a better analogy since they produce something that humans can use.
- allturtles 2y ago
  > There’s a large supply chain that AI is dependent on that requires humans to function. ...for now. Given sufficient advances in robotics, why would you expect that to continue?
- TeMPOraL 2y ago
  > Bees might be a better analogy since they produce something that humans can use. And yet they're endangered, and we already figured out how to do pollination, so we know we can survive without them - it's just going to be a huge pain. Some famines may follow, but likely not enough to endanger civilization as a whole. Thus even with this analogy, if humans end up being an annoying supply chain dependency to an AI, the AI might eventually work out an alternative supply chain, at which point we're back to being just an annoyance.
  - ben_w 2y ago
    > Some famines may follow, but likely not enough to endanger civilization as a whole. I'm not confident enough to rely on that: most people in the west have never encountered a famine, only much milder things like the price of one or two staples being high — eggs currently — never all of them at once. What will we do to ourselves if we face a famine? Will we go to war (or exterminate the local "undesirables") like the old days? How fragile are we now, compared to the last time that happened? How much has specialisation meant that the elimination of certain minorities will just break everything? "Furries run the internet", as the memes say. What other sectors are over-represented by a small minority?

michaelt 2y ago

> I dont understand, why any highly sophisticated AI should invest that much resources to kill us Well you see, everyone knows The Terminator and The Matrix and Frankenstein and The Golem of Prague and Rossum's Universal Robots. All of which share a theme: the sinful hubris of playing god and trying to create life will inevitably lead to us being struck down by the very being we created. In parallel, all the members of our educated classes have received philosophy education saying "utilitarianism says it's good to reduce total human suffering, but technically if you eliminated all humans there would be no suffering any more, ha ha obviously that's a reductio ad absurdum to show a weakness of utilitarianism please don't explode the world" And so in the Western cultural tradition, and especially among the sort of people who call themselves futurists Arnold Schwarzenegger firing a minigun is the defining image of AI.

ben_w 2y ago
I wouldn't categorise The Matrix or Frankenstein like that. The Matrix had humanity under control, but the machines had no desire to eliminate humanity, the machines just wanted to live — humans kept on fighting the machines even when the machines gave humanity an experiential paradise to live in. Frankenstein is harder because of how the book differs from the films. Your point it is valid because it is about the cultural aspects and I expect more have seen one of the films than to have read/listened to the book — but in the book, Adam was described as beautiful in every regard save for his eyes, he was a sensitive, emotional vegetarian, and he only learned anger after being consistently shown hatred and violence by absolutely everyone he ever met except that one who was blind.

achierius 2y ago

We did go out and exterminate (almost) all wolves because, yes, they would kill us while we were out and about. We also do happily gas/poison/fill-with-molten-aluminum entire nests of ants, not because they're killing us, but just because they're eating our food / for fun. And even when we didn't mean to -- how many species have we pushed to the brink just because we wanted to build cities where they happened to live? What happens when some AI wants to use your groundwater for its cooling system? It wouldn't be personal, but you'd starve to death regardless.