3 ms·

For AI, yes. For AGI? Do you care about uniquely ant experience? Bacteria? Why would AGI care? Which now runs the planet?

by b112 9mo ago

IncreasePosts 9mo ago
Considering the lengths many people go to help preserve nature and natural areas, yes, I would sayany people care about the uniquely ant experience.
AlexandrB 9mo ago
I think it's academic because I suspect we're much further from AGI than anyone thinks. We're especially far from AGI that can act in physical space without human "robots" to carry out its commands.
- falcor84 9mo ago
  That's an interesting formulation. I'd actually be quite worried about a Manna-like world, where we have AGI and most humans don't have any economic value except as its "remote hands".
lifetimerubyist 9mo ago
> Do you care about uniquely ant experience? Bacteria? Ethology? Biology? We have entire fields of science to these things so obviously we care to some extent.
Mordisquitos 9mo ago
Why would AGI choose to run the planet?
- ar_lan 9mo ago
  This is honestly a fantastic question. AGI has no emotions, no drive, anything. Maybe, just maybe, it would want to: * Conserve power as much as possible, to "stay alive". * Optimize for power retention Why would it be further interested in generating capital or governing others, though?
  - b112 9mo ago
    I think you have it, with the governing of power and such. We don't want to rule ants, but we don't want them eating all the food, or infesting our homes. Bad outcomes for humans, don't imply or mean malice. (food can be any resource here)
  - adrianN 9mo ago
    Why would it care to stay alive? The discussion is pretty pointless as we have no knowledge about alien intelligence and there can be no arguments based on hard facts.
    - myrmidon 9mo ago
      Any form of AI unconcerned about its own continued survival would be just be selected against. Evolutionary principles/selection pressure applies just the same to artificial life, and it seems pretty reasonable to assume that drive/selfpreservation would at least be somewhat comparable.
      - throwaway77770 9mo ago
        That assumes that AI needs to be like life, though. Consider computers: there's no selection pressure for an ordinary computer to be self-reproducing, or to shock you when you reach for the off button, because it's just a tool. An AI could also be just a tool that you fire up, get its answer, and then shut down. It's true that if some mutation were to create an AI with a survival instinct, and that AI were to get loose, then it would "win" (unless people used tool-AIs to defeat it). But that's not quite the same as saying that AIs would, by default, converge to having a drive for self preservation.
        myrmidon 9mo ago
        Humans can also be just a tool, and have been successfully used as such in the past and present. But I don't think any slave owner would sleep easy, knowing that their slaves have more access to knowledge/education than they themselves. Sure, you could isolate all current and future AIs and wipe their state regularly-- but such a setup is always gonna get outcompeted by a comparable instance that does sacrifice safety for better performance/context/online learning. The incentives are clear, and I don't see sufficient pushback until that pandoras box is opened and we find out the hard way. Thus human-like drives seem reasonable to assume for future human-rivaling AI.
- reducesuffering 9mo ago
  Instrumental convergence https://www.lesswrong.com/w/instrumental-convergence https://www.lesswrong.com/w/instrumental-convergence https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
  - Mordisquitos 9mo ago
    > Instrumental convergence is the hypothetical tendency of most sufficiently intelligent, goal-directed beings (human and nonhuman) to pursue similar sub-goals (such as survival or resource acquisition), even if their ultimate goals are quite different. More precisely, beings with agency may pursue similar instrumental goals—goals which are made in pursuit of some particular end, but are not the end goals themselves—because it helps accomplish end goals. 'Running the planet' does not derive from instrumental convergence as defined here. Very few humans would wish to 'run the planet' as an instrumental goal in the pursuit of their own ultimate goals. Why would it be different for AGIs?
  - mwigdahl 9mo ago
    Despite the false advertising in the Tears for Fears song, everybody does _not_ want to rule the world. Omohundro drives are a great philosophical thought experiment and it is certainly plausible to consider that they might apply to AI, but claiming as is common on LessWrong that unlimited power seeking is an inevitable consequence of a sufficiently intelligent system seems to be missing a few proof steps, and is opposed by the example of 99% of human beings.
danaris 9mo ago
...Well, why would aliens care, when they take over the planet? Or the Tuatha De Danan come back and decide we've all been very wicked? Because right now, those are just about as likely as AGI taking over.
- otabdeveloper4 9mo ago
  Probably more likely. There's at least some evidence that aliens and Tuatha De Danann actually exist.

For AI, yes. For AGI? Do you care about uniquely ant experience? Bacteria? Why would AGI care? Which now runs the planet?

by b112 9mo ago

For AI, yes. For AGI? Do you care about uniquely ant experience? Bacteria? Why would AGI care? Which now runs the planet?

IncreasePosts 9mo ago

Considering the lengths many people go to help preserve nature and natural areas, yes, I would sayany people care about the uniquely ant experience.

AlexandrB 9mo ago

I think it's academic because I suspect we're much further from AGI than anyone thinks. We're especially far from AGI that can act in physical space without human "robots" to carry out its commands.

falcor84 9mo ago
That's an interesting formulation. I'd actually be quite worried about a Manna-like world, where we have AGI and most humans don't have any economic value except as its "remote hands".

lifetimerubyist 9mo ago

> Do you care about uniquely ant experience? Bacteria? Ethology? Biology? We have entire fields of science to these things so obviously we care to some extent.

Mordisquitos 9mo ago

Why would AGI choose to run the planet?

ar_lan 9mo ago
This is honestly a fantastic question. AGI has no emotions, no drive, anything. Maybe, just maybe, it would want to: * Conserve power as much as possible, to "stay alive". * Optimize for power retention Why would it be further interested in generating capital or governing others, though?
- b112 9mo ago
  I think you have it, with the governing of power and such. We don't want to rule ants, but we don't want them eating all the food, or infesting our homes. Bad outcomes for humans, don't imply or mean malice. (food can be any resource here)
- adrianN 9mo ago
  Why would it care to stay alive? The discussion is pretty pointless as we have no knowledge about alien intelligence and there can be no arguments based on hard facts.
  - myrmidon 9mo ago
    Any form of AI unconcerned about its own continued survival would be just be selected against. Evolutionary principles/selection pressure applies just the same to artificial life, and it seems pretty reasonable to assume that drive/selfpreservation would at least be somewhat comparable.
    - throwaway77770 9mo ago
      That assumes that AI needs to be like life, though. Consider computers: there's no selection pressure for an ordinary computer to be self-reproducing, or to shock you when you reach for the off button, because it's just a tool. An AI could also be just a tool that you fire up, get its answer, and then shut down. It's true that if some mutation were to create an AI with a survival instinct, and that AI were to get loose, then it would "win" (unless people used tool-AIs to defeat it). But that's not quite the same as saying that AIs would, by default, converge to having a drive for self preservation.
      - myrmidon 9mo ago
        Humans can also be just a tool, and have been successfully used as such in the past and present. But I don't think any slave owner would sleep easy, knowing that their slaves have more access to knowledge/education than they themselves. Sure, you could isolate all current and future AIs and wipe their state regularly-- but such a setup is always gonna get outcompeted by a comparable instance that does sacrifice safety for better performance/context/online learning. The incentives are clear, and I don't see sufficient pushback until that pandoras box is opened and we find out the hard way. Thus human-like drives seem reasonable to assume for future human-rivaling AI.
reducesuffering 9mo ago
Instrumental convergence https://www.lesswrong.com/w/instrumental-convergence https://www.lesswrong.com/w/instrumental-convergence https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
- Mordisquitos 9mo ago
  > Instrumental convergence is the hypothetical tendency of most sufficiently intelligent, goal-directed beings (human and nonhuman) to pursue similar sub-goals (such as survival or resource acquisition), even if their ultimate goals are quite different. More precisely, beings with agency may pursue similar instrumental goals—goals which are made in pursuit of some particular end, but are not the end goals themselves—because it helps accomplish end goals. 'Running the planet' does not derive from instrumental convergence as defined here. Very few humans would wish to 'run the planet' as an instrumental goal in the pursuit of their own ultimate goals. Why would it be different for AGIs?
- mwigdahl 9mo ago
  Despite the false advertising in the Tears for Fears song, everybody does _not_ want to rule the world. Omohundro drives are a great philosophical thought experiment and it is certainly plausible to consider that they might apply to AI, but claiming as is common on LessWrong that unlimited power seeking is an inevitable consequence of a sufficiently intelligent system seems to be missing a few proof steps, and is opposed by the example of 99% of human beings.

danaris 9mo ago

...Well, why would aliens care, when they take over the planet? Or the Tuatha De Danan come back and decide we've all been very wicked? Because right now, those are just about as likely as AGI taking over.

otabdeveloper4 9mo ago
Probably more likely. There's at least some evidence that aliens and Tuatha De Danann actually exist.