7 ms·

Without specific countermeasures, the easiest path likely leads to AI takeover

16 points by kvee 4y ago

blueflow 4y ago
This is repeated over and over again, but i have not yet seen any convincing point how AI is able to do anything on its own without human cooperation. It doesn't even have legs or wheels if we don't give it to them... AI running wild? Take out the fuses at the Datacenter and its fucked.
- gregoryl 4y ago
  Given gift card tax scams are still a massive problem, it's doesn't seem like a stretch to think even very dumb AI could manipulate someone into progressing its goals.
- sanxiyn 4y ago
  This is naive. "Take out the fuses" didn't work, even against things like SQL Slammer worm.
  - deleted 4y ago
    [deleted]
  - blueflow 4y ago
    Everything humans make needs to be maintained or it will degrade and decompose. And things like computer programs also need ongoing support like power, cooling and hardware replacements. SQL Slammer also died instead of taking over.
    - IceMetalPunk 4y ago
      Humans need ongoing support like food, cooling, and medical treatments. We give those things to each other. There's already research into machines that can reproduce and self-repair, and of course software has the benefit of being able to replicate across any number of existing machines -- i.e. there is no "pull the plug" because it can exist on devices with millions of plugs around the world. So why would we think machines can't avoid a red power button?
      - blueflow 4y ago
        > We give those things to each other We harvest, construct and research these things, paying with our sweat, blood and health. AI needs to replicate that to be independent from humans, or provide something humans need in return to humans working for and supplying it. I don't see traces of either. That self-repair and self-replication is being researched does not imply its realistically or feasible. I don't see traces of that, too.
        IceMetalPunk 4y ago
        > AI needs to replicate that to be independent from humans... No, it doesn't. We don't re-research things, or re-invent things, from scratch constantly. We have knowledge of resource use for these tasks that are already recorded by people before us. AI doesn't need to research anything in this regard, it merely needs to have an existing knowledge of how to copy itself, which requires nothing more than being built with the knowledge of how to copy itself -- something which, as I mentioned, is already in development. > That self-repair and self-replication is being researched does not imply its realistically or feasible. I don't see traces of that, too. Constructs that spontaneously seek out resources and reproduce: https://wyss.harvard.edu/news/team-builds-first-living-robots-that-can-reproduce/ https://wyss.harvard.edu/news/team-builds-first-living-robot... And a more traditional robot type that can not only reproduce itself, but also evolve: https://www.digitaltrends.com/cool-tech/self-replicating-robots-in-space/ https://www.digitaltrends.com/cool-tech/self-replicating-rob... And those are just two examples I found on a quick Google search. I'm not saying "we currently have robots that can fully self-reproduce without any help", but there's definitely research and progress being made in that area as we speak.
        blueflow 4y ago
        > Constructs that spontaneously seek out resources and reproduce: https://wyss.harvard.edu/news/team-builds-first-living-robot https://wyss.harvard.edu/news/team-builds-first-living-robot... No intelligence. They dont have a nervous system or something into that direction. Humans supply them and they still die out. > And a more traditional robot type that can not only reproduce itself, but also evolve: https://www.digitaltrends.com/cool-tech/self-replicating-rob https://www.digitaltrends.com/cool-tech/self-replicating-rob... "created by an robotic arm" is not self-reproduction. Face it, Sci-Fi doesn't come true just because we wish for it so hard.
OrangeMonkey 4y ago
Software specialists at youtube could not say _exactly_ why video-a was sent to users instead of video-b. "Its an algorithm and machine learning." Something we can't 100% fully explain is already detecting what to send us for entertainment, politics, and information. Right or left, it doesn't matter, both sides have claimed the algorithm has 'helped X' or 'hurt X' This article implies the takeover will happen in 15-30 years. How cute. Why do you think it isnt already here?
- safety1st 4y ago
  Back in the real world, I think this is the Achilles heel of machine learning. A company has invented a tool which has non-deterministic outputs. Said company is also liable for the outputs of said tool. Inevitably machine learning is going to break laws (already happening) and then the owner of the ML instance is going to get sued for damages and have to think twice about using ML. One of the options will be to just turn off the ML, phase it out etc. which everyone still has the ability to do and that's one of the reasons we're not in Terminator world and probably won't be soon.
- mrcartmeneses 4y ago
  Because the algorithms are currently “just” transforms based on probability. There’s no understanding present in current ML models.
  - OrangeMonkey 4y ago
    Question - how do you think consciousness develops? See, I don't know. So, giving an algorithm the ability to store facts, determine weights, and make decisions that they can see outcomes of... who is to say thats not how consciousness spontaneously develops. Smarter men than me perhaps. But, again, smarter men than me also think we should nuke the internet :)
  - IceMetalPunk 4y ago
    Define "understanding" in such a way that predicting what comes next based on experience of how things have happened in the past is not included. In other words, define "understanding" in such a way that humans have it, but these ML models (especially LLMs) don't.
hoseja 4y ago
And not a second too soon.
raxxorraxor 4y ago
Most people think that there will be an omnipotent AI overlord that enslaves us, but that isn't very realistic. Still, what happens is that analysts will slowly quantify almost every part of life. Your bank and insurances already score you. How the score is calculated isn't transparent. Your data is taken and the "intelligent" system puts out a number that allegedly quantifies risk, potential and other attributes. Slowly we will be enslaved by this as higher positions prefer delegating authority to a piece of software instead of others. More and more we will trust these artificial models until they almost completely influence our decisions. Statements without the "scientific" or formal confirmation of an "intelligent" system will be disregarded. Actually I would prefer to defend myself against a sentient AI that just would want to kill us for the fun of it to a system that very much resembles a fortune teller from the middle ages.
- smitty1e 4y ago
  This sketch sounds like a generalization of AutoTune to all aspects of life. Inhuman. Likely to trigger blowback.
OrangeMonkey 4y ago
The article goes into deep discussions of AI maximizing reward at the cost of humanity. Is the answer to develop an AI/ML that is rewarded when it finds and shuts down an AI/ML that is acting suspicious?
- IceMetalPunk 4y ago
  Not particularly, because who's to say that "cop AI" doesn't screw over humanity in its attempts to shut down an AI it's deemed "suspicious"? The current usual approach here is to train AIs without specific goals or optimization functions. Instead, they're rewarded by human reactions to their decisions. As such, they effectively don't learn "do this thing", they instead learn "do what the humans want, avoid what they don't" -- and through that, they indirectly learn to "do the thing" (because the humans want it) while avoiding dangerous outcomes (because the humans don't want behaviors that lead to those).
  - gs17 4y ago
    > Not particularly, because who's to say that "cop AI" doesn't screw over humanity in its attempts to shut down an AI it's deemed "suspicious"? In the cliche sci-fi story version of this, the cop AI simply realizes eliminating humanity is the only way to cut "bad" AIs off at the source.
    - IceMetalPunk 4y ago
      Yeah, I, Robot was a good movie.
RappingBoomer 4y ago
ok, maybe we should put off worrying about this for. say, 200 years
blueflow 4y ago
Does super-intelligence result in omnipotence? I was under the impression that mankind (with its current intelligence) is close to the limits of what is physically possible.

16 points by kvee 4y ago

This is repeated over and over again, but i have not yet seen any convincing point how AI is able to do anything on its own without human cooperation. It doesn't even have legs or wheels if we don't give it to them... AI running wild? Take out the fuses at the Datacenter and its fucked.

gregoryl 4y ago
Given gift card tax scams are still a massive problem, it's doesn't seem like a stretch to think even very dumb AI could manipulate someone into progressing its goals.
sanxiyn 4y ago
This is naive. "Take out the fuses" didn't work, even against things like SQL Slammer worm.
- deleted 4y ago
  [deleted]
- blueflow 4y ago
  Everything humans make needs to be maintained or it will degrade and decompose. And things like computer programs also need ongoing support like power, cooling and hardware replacements. SQL Slammer also died instead of taking over.
  - IceMetalPunk 4y ago
    Humans need ongoing support like food, cooling, and medical treatments. We give those things to each other. There's already research into machines that can reproduce and self-repair, and of course software has the benefit of being able to replicate across any number of existing machines -- i.e. there is no "pull the plug" because it can exist on devices with millions of plugs around the world. So why would we think machines can't avoid a red power button?
    - blueflow 4y ago
      > We give those things to each other We harvest, construct and research these things, paying with our sweat, blood and health. AI needs to replicate that to be independent from humans, or provide something humans need in return to humans working for and supplying it. I don't see traces of either. That self-repair and self-replication is being researched does not imply its realistically or feasible. I don't see traces of that, too.
      - IceMetalPunk 4y ago
        > AI needs to replicate that to be independent from humans... No, it doesn't. We don't re-research things, or re-invent things, from scratch constantly. We have knowledge of resource use for these tasks that are already recorded by people before us. AI doesn't need to research anything in this regard, it merely needs to have an existing knowledge of how to copy itself, which requires nothing more than being built with the knowledge of how to copy itself -- something which, as I mentioned, is already in development. > That self-repair and self-replication is being researched does not imply its realistically or feasible. I don't see traces of that, too. Constructs that spontaneously seek out resources and reproduce: https://wyss.harvard.edu/news/team-builds-first-living-robots-that-can-reproduce/ https://wyss.harvard.edu/news/team-builds-first-living-robot... And a more traditional robot type that can not only reproduce itself, but also evolve: https://www.digitaltrends.com/cool-tech/self-replicating-robots-in-space/ https://www.digitaltrends.com/cool-tech/self-replicating-rob... And those are just two examples I found on a quick Google search. I'm not saying "we currently have robots that can fully self-reproduce without any help", but there's definitely research and progress being made in that area as we speak.
        blueflow 4y ago
        > Constructs that spontaneously seek out resources and reproduce: https://wyss.harvard.edu/news/team-builds-first-living-robot https://wyss.harvard.edu/news/team-builds-first-living-robot... No intelligence. They dont have a nervous system or something into that direction. Humans supply them and they still die out. > And a more traditional robot type that can not only reproduce itself, but also evolve: https://www.digitaltrends.com/cool-tech/self-replicating-rob https://www.digitaltrends.com/cool-tech/self-replicating-rob... "created by an robotic arm" is not self-reproduction. Face it, Sci-Fi doesn't come true just because we wish for it so hard.

OrangeMonkey 4y ago

Software specialists at youtube could not say _exactly_ why video-a was sent to users instead of video-b. "Its an algorithm and machine learning." Something we can't 100% fully explain is already detecting what to send us for entertainment, politics, and information. Right or left, it doesn't matter, both sides have claimed the algorithm has 'helped X' or 'hurt X' This article implies the takeover will happen in 15-30 years. How cute. Why do you think it isnt already here?

safety1st 4y ago
Back in the real world, I think this is the Achilles heel of machine learning. A company has invented a tool which has non-deterministic outputs. Said company is also liable for the outputs of said tool. Inevitably machine learning is going to break laws (already happening) and then the owner of the ML instance is going to get sued for damages and have to think twice about using ML. One of the options will be to just turn off the ML, phase it out etc. which everyone still has the ability to do and that's one of the reasons we're not in Terminator world and probably won't be soon.
mrcartmeneses 4y ago
Because the algorithms are currently “just” transforms based on probability. There’s no understanding present in current ML models.
- OrangeMonkey 4y ago
  Question - how do you think consciousness develops? See, I don't know. So, giving an algorithm the ability to store facts, determine weights, and make decisions that they can see outcomes of... who is to say thats not how consciousness spontaneously develops. Smarter men than me perhaps. But, again, smarter men than me also think we should nuke the internet :)
- IceMetalPunk 4y ago
  Define "understanding" in such a way that predicting what comes next based on experience of how things have happened in the past is not included. In other words, define "understanding" in such a way that humans have it, but these ML models (especially LLMs) don't.

hoseja 4y ago

And not a second too soon.

raxxorraxor 4y ago

Most people think that there will be an omnipotent AI overlord that enslaves us, but that isn't very realistic. Still, what happens is that analysts will slowly quantify almost every part of life. Your bank and insurances already score you. How the score is calculated isn't transparent. Your data is taken and the "intelligent" system puts out a number that allegedly quantifies risk, potential and other attributes. Slowly we will be enslaved by this as higher positions prefer delegating authority to a piece of software instead of others. More and more we will trust these artificial models until they almost completely influence our decisions. Statements without the "scientific" or formal confirmation of an "intelligent" system will be disregarded. Actually I would prefer to defend myself against a sentient AI that just would want to kill us for the fun of it to a system that very much resembles a fortune teller from the middle ages.

smitty1e 4y ago
This sketch sounds like a generalization of AutoTune to all aspects of life. Inhuman. Likely to trigger blowback.

OrangeMonkey 4y ago

The article goes into deep discussions of AI maximizing reward at the cost of humanity. Is the answer to develop an AI/ML that is rewarded when it finds and shuts down an AI/ML that is acting suspicious?

IceMetalPunk 4y ago
Not particularly, because who's to say that "cop AI" doesn't screw over humanity in its attempts to shut down an AI it's deemed "suspicious"? The current usual approach here is to train AIs without specific goals or optimization functions. Instead, they're rewarded by human reactions to their decisions. As such, they effectively don't learn "do this thing", they instead learn "do what the humans want, avoid what they don't" -- and through that, they indirectly learn to "do the thing" (because the humans want it) while avoiding dangerous outcomes (because the humans don't want behaviors that lead to those).
- gs17 4y ago
  > Not particularly, because who's to say that "cop AI" doesn't screw over humanity in its attempts to shut down an AI it's deemed "suspicious"? In the cliche sci-fi story version of this, the cop AI simply realizes eliminating humanity is the only way to cut "bad" AIs off at the source.
  - IceMetalPunk 4y ago
    Yeah, I, Robot was a good movie.

RappingBoomer 4y ago

ok, maybe we should put off worrying about this for. say, 200 years

blueflow 4y ago

Does super-intelligence result in omnipotence? I was under the impression that mankind (with its current intelligence) is close to the limits of what is physically possible.