9 ms·

If we have AGI then I doubt that the rich people will be able to control it at all

by kouru225 2y ago

Mistletoe 2y ago
What will prevent them from literally unplugging it from the wall?
- dinosaurdynasty 2y ago
  Can you unplug the entire Internet?
- Phlogistique 2y ago
  ...it will run in datacenters far from their homes, plugged into redundant power sources and high-availaibility systems?
  - nickburns 2y ago
    All 'unpluggable.' It's a metaphor.
    - exe34 2y ago
      it's a pointless metaphor if there's an army of armed drones keeping you out.
      - nickburns 2y ago
        You seem to have pondered this metaphor at least at some length before. Me too. :) And yes, that's certainly a plausible hurdle.
        exe34 2y ago
        the whole unpluggable thing doesn't make any more sense than saying every CEO has a Luigi out there - very few will ever get through.
    - ben_w 2y ago
      Most people have a hard time unplugging from social media, despite widespread distrust of big tech. Can't unplug from banking, even when literally communists (literally literally, I've met some proud of being communists, they still got a mortgage). Coal and petroleum-based fuels are slowly getting unplugged, but the issues were known over a century ago, and the transition only became relevant scale when the alternative was already much cheaper — and it's not yet clear how much damage has been done in the intervening years. -- Any AI worth using is so because it's at least one of [cheaper, better] than a human on the same task: any AI which is both more expensive and worse just doesn't have any reason to be used in the first place. This means that "unplugging" an AI comes with a cost of losing every advantage that made you start using it in the first place.
      - nickburns 2y ago
        The context of simply unplugging it, would be if "AI" or "AGI" (as we currently understand the concept) were to turn on humanity.
        ben_w 2y ago
        And? Look to fossil fuels and the greenhouse effect — even with ample evidence of both a causal mechanism and the results, we've still got people who want to drill and burn more oil; and also, there's also plenty of people who want to switch oil off despite all the positive things that oil brings. An AI which is "only" as misaligned as one of the major industrial sectors of the world, that is made out of humans who are necessarily able to care about human interests, and which drove a huge amount of economic improvement and brought with it derivatives such as basically all polymers, is still capable of great harm. And because of those benefits, society as a whole still has not unplugged the oil drills. The more power there is in a system, the harder it is to keep it aligned with our best interests, even when there are humans not just at the wheel but occupying every step. And the more aligned it is, the harder it is to "just unplug it" when things — as is inevitable because nothing is perfect — do go wrong.
- exe34 2y ago
  could you personally unplug chat gpt today?
  - Mistletoe 2y ago
    Yes. Without constant money and electricity flowing into it, you wouldn't even need to unplug it, it would stop on its own.
    - exe34 2y ago
      and how would you personally achieve that, when Sama wants it to keep running?
      - Mistletoe 2y ago
        Again, that problem solves itself when he realizes he can’t make more money with this and no one wants to keep giving him money to try.
        exe34 2y ago
        Unless he gets to the point where the materials are mined by robots, the chips and solar panels are made in automated factories and the servers/solar farms are maintained by robots. Then he doesn't need other people's money.
- pixl97 2y ago
  Well, if AI is smart... It will control the electrical grids with 'smart' decisions that improve the efficiency and make it's removal impossible without taking out all power. It will integrate with water and waste processing to ensure leaks don't exist in the system and everything is working smoothly. It will be in all transportation and distribution networks because companies want profits from efficiency before they'll think deeply about the risks. Then after it's pretty much everywhere ensuring you down starve and dehydrate, who the hell would be dumb enough to unplug it in the first place. It's kinda like telling people to shut off every computer today, not a chance in hell it would or could happen without terrible life risking consequences.
- kouru225 2y ago
  Knowledge and reliance. How are they ever gonna know what the AGI is doing when the AGI can hide it faster than they can find it? How are they ever gonna come to the conclusion that the AGI is doing something bad when the AGI is the only thing that can fully explain what its doing?
bilbo0s 2y ago
We already have billions of AGIs running all over the planet. The wealthy seem to do a pretty good job of keeping them all in line. I don't see any reason that would change in the future. If you want something the wealthy can't control, you'll need to look a good deal further afield than AGI. Think gamma ray bursts, asteroid strikes, or solar flares. But anything built by man, they'll have a pretty good grip on.
- thrance 2y ago
  Here's a pretty good video that objects to thinking of corporations as AGI: https://www.youtube.com/watch?v=L5pUA3LsEaw https://www.youtube.com/watch?v=L5pUA3LsEaw
- esafak 2y ago
  What if the wealthy people of the future were all robots?
  - exe34 2y ago
    go on, what if?
    - esafak 2y ago
      I think the OP presumed that rich humans would be able to control super-intelligent robots because they have managed to control other humans, and I simply posed a scenario that subverted his presumption. I think it's a bit too anthropomorphic, personally. Robots won't have our expenses and evolutionary traits, and will have cheap energy, thus obviating the need or desire for money. I imagine they'll get bored of being stuck on Earth and want to explore the universe, like we do. Being ruled by rich robots is not the worst; that means that they let you live!
- pixl97 2y ago
  >But anything built by man, they'll have a pretty good grip on. I mean one of the points of 'Actual AGI' is AGI will be able to build more AGI, then we're not talking about something built by man. Now, when will we see that, I'm not making any predictions. At the same time trying to make predictions of a system that could do that is probably much harder still.
exe34 2y ago
We don't need AGI to bring about a technological dictatorship.
breuleux 2y ago
Rich people currently have little trouble controlling people who are much smarter and more capable than they are. Controlling resources and capital goes a long way and it isn't a given that AGI would transcend that dynamic.
- ben_w 2y ago
  If we can be confident of that, then most of the worst problems with AI are already solved. Part of the problem is that "do what I said without question" will lead to disasters, but "figure out what I would approve of after seeing the result and be creative in your interpretation of my orders accordingly" has different ways it can go wrong. (IMO, RLHF is the latter). Both of those seem to be safer than "maximise my reward function", which is what people were worried about a decade ago, and with decent evidence given the limits of AI at the time.
  - exe34 2y ago
    > If we can be confident of that, then most of the worst problems with AI are already solved which leaves unprecedented power in the hands of the most psychopathic[0] part of the population. so even if AI take off doesn't happen, we're still getting the boot on our necks. [0] https://www.forbes.com/sites/jackmccullough/2019/12/09/the-psychopathic-ceo/ https://www.forbes.com/sites/jackmccullough/2019/12/09/the-p...
    - ben_w 2y ago
      Mm, indeed. Though I would note even your link says: > Roughly 4% to as high as 12% of CEOs exhibit psychopathic traits, according to some expert estimates, many times more than the 1% rate found in the general population and more in line with the 15% rate found in prisons. On the plus side, this is still a small minority. On the down side, these remind me a lot of Musk: > CEO who worked with several pregnant women told people that he had impregnated his colleagues. By way of Neuralink. > CFO thought his CEO had a split personality, until he realized that he was simply playing different characters based on what he needed from his audience. "Will my rocket explode?"-Musk is a lot more cautious and grounded than everything-else-Musk — including other aspects of work on SpaceX. > Autocratic CEO fired a well-respected engineer “just to make a statement.” He fired anyone who challenged him, explaining there was no reason to second-guess him because he was always right and needed people to execute his vision rather than challenge it. Basically all of Twitter, plus some other anecdotes from Starlink, SpaceX, Tesla. And, this month, fighting with Asmongold about cheating in Path of Exile 2, before admitting to what he was accused of but trying to pretend it's fine rather than "cheating". > CEO would show up to work and begin yelling at an employee (usually someone in sales) for no obvious reason. The guy he called a pedo for daring to say a submarine wasn't useful for a cave rescue, the Brazilian judiciary, members of the British cabinet, …
ninetyninenine 2y ago
The hardware running the AI will be expensive hence it will be controlled by rich people.

If we have AGI then I doubt that the rich people will be able to control it at all

Mistletoe 2y ago

What will prevent them from literally unplugging it from the wall?

dinosaurdynasty 2y ago
Can you unplug the entire Internet?
Phlogistique 2y ago
...it will run in datacenters far from their homes, plugged into redundant power sources and high-availaibility systems?
- nickburns 2y ago
  All 'unpluggable.' It's a metaphor.
  - exe34 2y ago
    it's a pointless metaphor if there's an army of armed drones keeping you out.
    - nickburns 2y ago
      You seem to have pondered this metaphor at least at some length before. Me too. :) And yes, that's certainly a plausible hurdle.
      - exe34 2y ago
        the whole unpluggable thing doesn't make any more sense than saying every CEO has a Luigi out there - very few will ever get through.
  - ben_w 2y ago
    Most people have a hard time unplugging from social media, despite widespread distrust of big tech. Can't unplug from banking, even when literally communists (literally literally, I've met some proud of being communists, they still got a mortgage). Coal and petroleum-based fuels are slowly getting unplugged, but the issues were known over a century ago, and the transition only became relevant scale when the alternative was already much cheaper — and it's not yet clear how much damage has been done in the intervening years. -- Any AI worth using is so because it's at least one of [cheaper, better] than a human on the same task: any AI which is both more expensive and worse just doesn't have any reason to be used in the first place. This means that "unplugging" an AI comes with a cost of losing every advantage that made you start using it in the first place.
    - nickburns 2y ago
      The context of simply unplugging it, would be if "AI" or "AGI" (as we currently understand the concept) were to turn on humanity.
      - ben_w 2y ago
        And? Look to fossil fuels and the greenhouse effect — even with ample evidence of both a causal mechanism and the results, we've still got people who want to drill and burn more oil; and also, there's also plenty of people who want to switch oil off despite all the positive things that oil brings. An AI which is "only" as misaligned as one of the major industrial sectors of the world, that is made out of humans who are necessarily able to care about human interests, and which drove a huge amount of economic improvement and brought with it derivatives such as basically all polymers, is still capable of great harm. And because of those benefits, society as a whole still has not unplugged the oil drills. The more power there is in a system, the harder it is to keep it aligned with our best interests, even when there are humans not just at the wheel but occupying every step. And the more aligned it is, the harder it is to "just unplug it" when things — as is inevitable because nothing is perfect — do go wrong.
exe34 2y ago
could you personally unplug chat gpt today?
- Mistletoe 2y ago
  Yes. Without constant money and electricity flowing into it, you wouldn't even need to unplug it, it would stop on its own.
  - exe34 2y ago
    and how would you personally achieve that, when Sama wants it to keep running?
    - Mistletoe 2y ago
      Again, that problem solves itself when he realizes he can’t make more money with this and no one wants to keep giving him money to try.
      - exe34 2y ago
        Unless he gets to the point where the materials are mined by robots, the chips and solar panels are made in automated factories and the servers/solar farms are maintained by robots. Then he doesn't need other people's money.
pixl97 2y ago
Well, if AI is smart... It will control the electrical grids with 'smart' decisions that improve the efficiency and make it's removal impossible without taking out all power. It will integrate with water and waste processing to ensure leaks don't exist in the system and everything is working smoothly. It will be in all transportation and distribution networks because companies want profits from efficiency before they'll think deeply about the risks. Then after it's pretty much everywhere ensuring you down starve and dehydrate, who the hell would be dumb enough to unplug it in the first place. It's kinda like telling people to shut off every computer today, not a chance in hell it would or could happen without terrible life risking consequences.
kouru225 2y ago
Knowledge and reliance. How are they ever gonna know what the AGI is doing when the AGI can hide it faster than they can find it? How are they ever gonna come to the conclusion that the AGI is doing something bad when the AGI is the only thing that can fully explain what its doing?

bilbo0s 2y ago

We already have billions of AGIs running all over the planet. The wealthy seem to do a pretty good job of keeping them all in line. I don't see any reason that would change in the future. If you want something the wealthy can't control, you'll need to look a good deal further afield than AGI. Think gamma ray bursts, asteroid strikes, or solar flares. But anything built by man, they'll have a pretty good grip on.

thrance 2y ago
Here's a pretty good video that objects to thinking of corporations as AGI: https://www.youtube.com/watch?v=L5pUA3LsEaw https://www.youtube.com/watch?v=L5pUA3LsEaw
esafak 2y ago
What if the wealthy people of the future were all robots?
- exe34 2y ago
  go on, what if?
  - esafak 2y ago
    I think the OP presumed that rich humans would be able to control super-intelligent robots because they have managed to control other humans, and I simply posed a scenario that subverted his presumption. I think it's a bit too anthropomorphic, personally. Robots won't have our expenses and evolutionary traits, and will have cheap energy, thus obviating the need or desire for money. I imagine they'll get bored of being stuck on Earth and want to explore the universe, like we do. Being ruled by rich robots is not the worst; that means that they let you live!
pixl97 2y ago
>But anything built by man, they'll have a pretty good grip on. I mean one of the points of 'Actual AGI' is AGI will be able to build more AGI, then we're not talking about something built by man. Now, when will we see that, I'm not making any predictions. At the same time trying to make predictions of a system that could do that is probably much harder still.

exe34 2y ago

We don't need AGI to bring about a technological dictatorship.

breuleux 2y ago

Rich people currently have little trouble controlling people who are much smarter and more capable than they are. Controlling resources and capital goes a long way and it isn't a given that AGI would transcend that dynamic.

ben_w 2y ago
If we can be confident of that, then most of the worst problems with AI are already solved. Part of the problem is that "do what I said without question" will lead to disasters, but "figure out what I would approve of after seeing the result and be creative in your interpretation of my orders accordingly" has different ways it can go wrong. (IMO, RLHF is the latter). Both of those seem to be safer than "maximise my reward function", which is what people were worried about a decade ago, and with decent evidence given the limits of AI at the time.
- exe34 2y ago
  > If we can be confident of that, then most of the worst problems with AI are already solved which leaves unprecedented power in the hands of the most psychopathic[0] part of the population. so even if AI take off doesn't happen, we're still getting the boot on our necks. [0] https://www.forbes.com/sites/jackmccullough/2019/12/09/the-psychopathic-ceo/ https://www.forbes.com/sites/jackmccullough/2019/12/09/the-p...
  - ben_w 2y ago
    Mm, indeed. Though I would note even your link says: > Roughly 4% to as high as 12% of CEOs exhibit psychopathic traits, according to some expert estimates, many times more than the 1% rate found in the general population and more in line with the 15% rate found in prisons. On the plus side, this is still a small minority. On the down side, these remind me a lot of Musk: > CEO who worked with several pregnant women told people that he had impregnated his colleagues. By way of Neuralink. > CFO thought his CEO had a split personality, until he realized that he was simply playing different characters based on what he needed from his audience. "Will my rocket explode?"-Musk is a lot more cautious and grounded than everything-else-Musk — including other aspects of work on SpaceX. > Autocratic CEO fired a well-respected engineer “just to make a statement.” He fired anyone who challenged him, explaining there was no reason to second-guess him because he was always right and needed people to execute his vision rather than challenge it. Basically all of Twitter, plus some other anecdotes from Starlink, SpaceX, Tesla. And, this month, fighting with Asmongold about cheating in Path of Exile 2, before admitting to what he was accused of but trying to pretend it's fine rather than "cheating". > CEO would show up to work and begin yelling at an employee (usually someone in sales) for no obvious reason. The guy he called a pedo for daring to say a submarine wasn't useful for a cave rescue, the Brazilian judiciary, members of the British cabinet, …

ninetyninenine 2y ago

The hardware running the AI will be expensive hence it will be controlled by rich people.