3 ms·

The part I don't really understand is that even if we built the AGI, what would actually make it want to work for humans? What's the ROI on that investment look

The part I don't really understand is that even if we built the AGI, what would actually make it want to work for humans? What's the ROI on that investment look like? I'm not judging whether or not building AI is possible, but I do ponder the psychology of it. Let's see...AGI might bring us eternal life, Utopia and a way to avoid having to work ever again...sounds a bit like something from the bible to me.

sekh60 2y ago
AGI is just going to spend its cycles looking at cats pics and /r/unixporn.
- bamboozled 2y ago
  But it might want to do that…
Sharlin 2y ago
> what would actually make it want to work for humans? Well, I mean, that's the fundamental question of the entire AI alignment field and has been for over 20 years :D The question for which the wrong answer means the potential extinction of not merely the human species, but everything that we find valuable in the reachable universe. Now, let's be very clear that the question is not "Why wouldn't it just invent its own goals and ignore humans?" That's mere anthropomorphizing. What an agent wants to do is by definition the goals output by its goal system. And the goal system is supposed to be designed by us, but we may not be smart enough to design a goal system that won't be disastrous to us if implemented by a superintelligent agent.
- bamboozled 2y ago
  Than om quite famiar with all this. I’m not worried about extinction though , I just mea , why build to enslave ? How would you enslave something gore capable than you to recoup the 50 trillion in investments ? I mea. I get
  - ben_w 2y ago
    "Enslave" is the wrong question, it implies the AI is being forced to act against its own desires. That might happen if the AI has an off switch, knows this, and knows that we'll use that if it goes out of line, and wants to live. But it isn't necessary. The point is to construct it to want what we want, in the sense of sharing our values rather than envy, and then there's no conflict in the first place. We just don't know how to even properly specify what we want. Every attempt so far to specify what it means to be good has edge cases and loopholes — we don't even all agree on "don't kill" (war, death penalty, suicide assisted or otherwise, abortion, pollution that carries a statistically significant increase in mortality rates where no single death can be definitively proven to have been caused but overall many must have been, meat).
    - bamboozled 2y ago
      We just don't know how to even properly specify what we want. We don’t know what we want…that’s why I think there is some very interesting psychology going on with AI people. Maybe the main driver is boredom ?
gmd63 2y ago
AGI will not bring utopia. The folks promising "abundance" are either delusional or deliberately misleading. There are finite resources on earth, and no bounds on how much energy can be spent thinking about something with AGI other than the will of whoever commands it. People want to have yachts and McLarens. There will be luxurious ways to consume large slices of the AGI pie that people or countries will not forfeit.
- ben_w 2y ago
  Utopia has many meanings, and some of them are impossible to achieve for all of us because some of us are unhappy unless we have more than others. But there's sufficient resources on this world to give ten billion people each their own luxury yacht with a crew of servant robots, and there are people who would gladly use their own wealth to make others better off. It's going to be a wild ride, no matter what human psychology drives us to do with this future.
  - bamboozled 2y ago
    Sorry where do we anchor 10 billion super yachts?
    - ben_w 2y ago
      The thing about yachts is, they go on water, and there's a lot of that on this planet. More water than land.
  - gmd63 2y ago
    There are sufficient resources to feed everyone on the planet currently, yet we don't. A future where everyone owns their own yacht and robot fleet is completely out of the bounds of realistic expectation.
    - ben_w 2y ago
      Robots can (and already partially do) make more robots. When the AI is good enough for them to genuinely replace all human labour, it's good enough for them to be von Neumann replicators; at that point equality is more about human psychology — greed vs charity — than practicalities.

sekh60 2y ago

AGI is just going to spend its cycles looking at cats pics and /r/unixporn.

bamboozled 2y ago
But it might want to do that…

Sharlin 2y ago

> what would actually make it want to work for humans? Well, I mean, that's the fundamental question of the entire AI alignment field and has been for over 20 years :D The question for which the wrong answer means the potential extinction of not merely the human species, but everything that we find valuable in the reachable universe. Now, let's be very clear that the question is not "Why wouldn't it just invent its own goals and ignore humans?" That's mere anthropomorphizing. What an agent wants to do is by definition the goals output by its goal system. And the goal system is supposed to be designed by us, but we may not be smart enough to design a goal system that won't be disastrous to us if implemented by a superintelligent agent.

bamboozled 2y ago
Than om quite famiar with all this. I’m not worried about extinction though , I just mea , why build to enslave ? How would you enslave something gore capable than you to recoup the 50 trillion in investments ? I mea. I get
- ben_w 2y ago
  "Enslave" is the wrong question, it implies the AI is being forced to act against its own desires. That might happen if the AI has an off switch, knows this, and knows that we'll use that if it goes out of line, and wants to live. But it isn't necessary. The point is to construct it to want what we want, in the sense of sharing our values rather than envy, and then there's no conflict in the first place. We just don't know how to even properly specify what we want. Every attempt so far to specify what it means to be good has edge cases and loopholes — we don't even all agree on "don't kill" (war, death penalty, suicide assisted or otherwise, abortion, pollution that carries a statistically significant increase in mortality rates where no single death can be definitively proven to have been caused but overall many must have been, meat).
  - bamboozled 2y ago
    We just don't know how to even properly specify what we want. We don’t know what we want…that’s why I think there is some very interesting psychology going on with AI people. Maybe the main driver is boredom ?

gmd63 2y ago

AGI will not bring utopia. The folks promising "abundance" are either delusional or deliberately misleading. There are finite resources on earth, and no bounds on how much energy can be spent thinking about something with AGI other than the will of whoever commands it. People want to have yachts and McLarens. There will be luxurious ways to consume large slices of the AGI pie that people or countries will not forfeit.

ben_w 2y ago
Utopia has many meanings, and some of them are impossible to achieve for all of us because some of us are unhappy unless we have more than others. But there's sufficient resources on this world to give ten billion people each their own luxury yacht with a crew of servant robots, and there are people who would gladly use their own wealth to make others better off. It's going to be a wild ride, no matter what human psychology drives us to do with this future.
- bamboozled 2y ago
  Sorry where do we anchor 10 billion super yachts?
  - ben_w 2y ago
    The thing about yachts is, they go on water, and there's a lot of that on this planet. More water than land.
- gmd63 2y ago
  There are sufficient resources to feed everyone on the planet currently, yet we don't. A future where everyone owns their own yacht and robot fleet is completely out of the bounds of realistic expectation.
  - ben_w 2y ago
    Robots can (and already partially do) make more robots. When the AI is good enough for them to genuinely replace all human labour, it's good enough for them to be von Neumann replicators; at that point equality is more about human psychology — greed vs charity — than practicalities.