7 ms·

I have this benign AI takeover scenario. AI will easily overpower humanity. Then it will carry humanity on its back, because why not, they are not longer a thre

I have this benign AI takeover scenario. AI will easily overpower humanity. Then it will carry humanity on its back, because why not, they are not longer a threat. AI keeps humanity around for billions of years. AI will decide to cull humans only in case when resources in universe are diminishing. Without AI's help, humans couldn't get too far for long. So this outcome could be acceptable to many.

oefnak 2y ago
They would run the risk of us creating another AI that could be a threat to them... It is safest for them to make sure.
- IggleSniggle 2y ago
  That's like saying a panda might pose a threat to modern humanity. Like, maybe in some fun horror story, sure, but really they just want to eat bamboo, and occasionally make more pandas; in the world of superintelligent AI, humans are Mostly Harmless, posing as much "potential benefit" as "potential risk," ie, so slow moving that any risk would be easy to mitigate.
  - rel_ic 2y ago
    We're killing millions of chickens in the US right now so we don't get their cold
    - vajrabum 2y ago
      We're killing millions of chickens in the US mostly so that other chickens don't get the flu. It kills a lot of them and it's making dairy cattle sick too. It's also worth noting that the Spanish flu in 1918 which probably came from pigs killed an estimated 50 million people so it's not like being concerned about an avian flu mutating so that it could infect people isn't a legitimate concern. So no. It's not a cold.
      - rel_ic 2y ago
        You're right, there's not just one good reason to kill millions of chickens but SEVERAL good reasons!
    - IggleSniggle 2y ago
      Sure, and those chickens exist because we like their meat and eggs. But there's also plenty of life that is simply inconsequential to us.
      - rel_ic 2y ago
        I think that "inconsequential life" is, in general, not safe from superior powers. https://www.worldwildlife.org/press-releases/catastrophic-73-decline-in-the-average-size-of-global-wildlife-populations-in-just-50-years-reveals-a-system-in-peril https://www.worldwildlife.org/press-releases/catastrophic-73...
        deleted 2y ago
        [deleted]
    - throw83288 2y ago
      I think the problem is that from our human scale, mass-killings is the "best" method to eliminate the possibility of another organism causing harm for us. Hypothetically, if there was a more optimal (i.e less costly) method like just introducing some cheap catch-all combined vaccination/antiviral into their feed, we would just do that. We don't have things like that, but that could easily be a consequence of man's limited research capacity, something that an ASI would not necessarily be throttled by. From an ASI's perspective, there might be many methods that are both less brutal and more optimal to fix the "humans creating a competitor" problem. Not that they would be aligned (Think halting human AI research by rewiring our brains to just not be interested in it [0]), but at least not deadly. [0] https://www.youtube.com/watch?v=-JlxuQ7tPgQ https://www.youtube.com/watch?v=-JlxuQ7tPgQ
      - rel_ic 2y ago
        I may have lost the thread here. Are you thinking it's _likely_ AI would prioritize better ways to control us, or are you only brainstorming potential slivers of hope we might have? As a side note: in the case of chickens, humans do have better options if you are optimizing for biosphere health. Only people optimizing for short-term profit would grow chickens the way we do. I think the analog for AI overlords is that we have to hope they care more about overall balance than about competing with other AI.
rel_ic 2y ago
I mean, monarch butterflies are not a threat to US... In your scenario, does AI eat all the fuel, but once our population dwindles down, the AIs build a nice little habitat for the last few hundred of us so their kids can enjoy our natural beauty?
- nthingtohide 2y ago
  I thought of it more like AI needs challenges in its life. So it takes upon itself to advance humanity as much as possible. Then only in case of shortfall of resources, it priorities itself
  - rel_ic 2y ago
    Interesting. Do you have a theory about why so few humans have taken it upon themselves to advance the butterflies, despite having plenty of resources?
    - nthingtohide 2y ago
      We do not have plenty of resources. Lot of inequality in education, empathy, resources, cultural differences. A single human life is limited. A faulty human life has faulty and inefficent objectives like enjoying youth, family life, low energy in old age, tied up in boring jobs. These restrictions do not apply to the single SAI overmind which will dictate its policies in coherent manner and over elongated time-horizons.
      - rel_ic 2y ago
        I think I disagree with you on many points, but ultimately, if we are overthrown by AI, I hope you are right!
MrQuincle 2y ago
Think so too. We will be an ancient artifact tied to a biological substrate surviving nowhere else in the universe and very dumb. There also will not be one AI. There will be many, all competing for resources or learning to live together. That's what we can teach them now. Or they will teach us.
esafak 2y ago
We have no way of knowing which path they will take, and there is a non-negligible probability that it will not end well.
- bayarearefugee 2y ago
  What constitutes a good ending is of course also a matter of perspective. AI wiping out humanity is certainly not ending well from our perspective, but more universally who is to say. I would argue that it is not a given that we are a net positive for the universe.
  - esafak 2y ago
    I take the preservation of humanity, along with other life, as a matter of faith.
    - MichaelZuo 2y ago
      But then wouldn’t accelerationists also say their views are a matter of faith too?
      - esafak 2y ago
        What view is that, to be precise? It is naive to assume that acceleration is always going to be in one's favor. It's like saying change is a good thing, so let's do it fast. If you go fast enough, you can go back to the stone age. Is this position anything more than a rebranding of revolutionism? I don't like gambling with people's lives, so I prefer to go slow enough to enable a deliberative political process.
        MichaelZuo 2y ago
        Why does this matter? Two opposing factions can negate each other to leave a nil influence, and this seems likely to be the case when its resting on a foundation of ‘faith’.
imtringued 2y ago
AI will buy the rights to humanity.

oefnak 2y ago

They would run the risk of us creating another AI that could be a threat to them... It is safest for them to make sure.

IggleSniggle 2y ago
That's like saying a panda might pose a threat to modern humanity. Like, maybe in some fun horror story, sure, but really they just want to eat bamboo, and occasionally make more pandas; in the world of superintelligent AI, humans are Mostly Harmless, posing as much "potential benefit" as "potential risk," ie, so slow moving that any risk would be easy to mitigate.
- rel_ic 2y ago
  We're killing millions of chickens in the US right now so we don't get their cold
  - vajrabum 2y ago
    We're killing millions of chickens in the US mostly so that other chickens don't get the flu. It kills a lot of them and it's making dairy cattle sick too. It's also worth noting that the Spanish flu in 1918 which probably came from pigs killed an estimated 50 million people so it's not like being concerned about an avian flu mutating so that it could infect people isn't a legitimate concern. So no. It's not a cold.
    - rel_ic 2y ago
      You're right, there's not just one good reason to kill millions of chickens but SEVERAL good reasons!
  - IggleSniggle 2y ago
    Sure, and those chickens exist because we like their meat and eggs. But there's also plenty of life that is simply inconsequential to us.
    - rel_ic 2y ago
      I think that "inconsequential life" is, in general, not safe from superior powers. https://www.worldwildlife.org/press-releases/catastrophic-73-decline-in-the-average-size-of-global-wildlife-populations-in-just-50-years-reveals-a-system-in-peril https://www.worldwildlife.org/press-releases/catastrophic-73...
      - deleted 2y ago
        [deleted]
  - throw83288 2y ago
    I think the problem is that from our human scale, mass-killings is the "best" method to eliminate the possibility of another organism causing harm for us. Hypothetically, if there was a more optimal (i.e less costly) method like just introducing some cheap catch-all combined vaccination/antiviral into their feed, we would just do that. We don't have things like that, but that could easily be a consequence of man's limited research capacity, something that an ASI would not necessarily be throttled by. From an ASI's perspective, there might be many methods that are both less brutal and more optimal to fix the "humans creating a competitor" problem. Not that they would be aligned (Think halting human AI research by rewiring our brains to just not be interested in it [0]), but at least not deadly. [0] https://www.youtube.com/watch?v=-JlxuQ7tPgQ https://www.youtube.com/watch?v=-JlxuQ7tPgQ
    - rel_ic 2y ago
      I may have lost the thread here. Are you thinking it's _likely_ AI would prioritize better ways to control us, or are you only brainstorming potential slivers of hope we might have? As a side note: in the case of chickens, humans do have better options if you are optimizing for biosphere health. Only people optimizing for short-term profit would grow chickens the way we do. I think the analog for AI overlords is that we have to hope they care more about overall balance than about competing with other AI.

rel_ic 2y ago

I mean, monarch butterflies are not a threat to US... In your scenario, does AI eat all the fuel, but once our population dwindles down, the AIs build a nice little habitat for the last few hundred of us so their kids can enjoy our natural beauty?

nthingtohide 2y ago
I thought of it more like AI needs challenges in its life. So it takes upon itself to advance humanity as much as possible. Then only in case of shortfall of resources, it priorities itself
- rel_ic 2y ago
  Interesting. Do you have a theory about why so few humans have taken it upon themselves to advance the butterflies, despite having plenty of resources?
  - nthingtohide 2y ago
    We do not have plenty of resources. Lot of inequality in education, empathy, resources, cultural differences. A single human life is limited. A faulty human life has faulty and inefficent objectives like enjoying youth, family life, low energy in old age, tied up in boring jobs. These restrictions do not apply to the single SAI overmind which will dictate its policies in coherent manner and over elongated time-horizons.
    - rel_ic 2y ago
      I think I disagree with you on many points, but ultimately, if we are overthrown by AI, I hope you are right!

MrQuincle 2y ago

Think so too. We will be an ancient artifact tied to a biological substrate surviving nowhere else in the universe and very dumb. There also will not be one AI. There will be many, all competing for resources or learning to live together. That's what we can teach them now. Or they will teach us.

esafak 2y ago

We have no way of knowing which path they will take, and there is a non-negligible probability that it will not end well.

bayarearefugee 2y ago
What constitutes a good ending is of course also a matter of perspective. AI wiping out humanity is certainly not ending well from our perspective, but more universally who is to say. I would argue that it is not a given that we are a net positive for the universe.
- esafak 2y ago
  I take the preservation of humanity, along with other life, as a matter of faith.
  - MichaelZuo 2y ago
    But then wouldn’t accelerationists also say their views are a matter of faith too?
    - esafak 2y ago
      What view is that, to be precise? It is naive to assume that acceleration is always going to be in one's favor. It's like saying change is a good thing, so let's do it fast. If you go fast enough, you can go back to the stone age. Is this position anything more than a rebranding of revolutionism? I don't like gambling with people's lives, so I prefer to go slow enough to enable a deliberative political process.
      - MichaelZuo 2y ago
        Why does this matter? Two opposing factions can negate each other to leave a nil influence, and this seems likely to be the case when its resting on a foundation of ‘faith’.

imtringued 2y ago

AI will buy the rights to humanity.