5 ms·

Okay, I would first be doubtful that we as good actors, could develop such a fool proof system. It would require just one mistake where the AI would have some s

Okay, I would first be doubtful that we as good actors, could develop such a fool proof system. It would require just one mistake where the AI would have some sort of network access, whether it's a security loophole, or social engineering type of thing, insider threat, etc. But let's talk about a more loose cannon type of case first. How could you be certain that other entities, or countries with desires for world domination wouldn't let the ASI loose?

alephnerd 2y ago
> But let's talk about a more loose cannon type of case first Let's talk about the resurrection of Christ, Avtar Kalki riding on a white horse, or the impending arrival of Trisolarians while we're at it. This is just techno-millenarianism that ignores very tangible and existential dangers with ML today - automated disinformation, PII leakage, hallucinations generating misinformation. AGI is extremely far away, and we anyhow have more closer term existential problems to worry about (and policymakers know that btw). And anyhow, even if AGI was a thing, you still need networks, electricity, and bullets - and humans can use that to shut stuff down. Sure it would hurt, but internet shutdowns and electricity shutdowns with shoot on site curfew orders are time tested solution to civil unrest if SHTF. Go to Urumqi in 2013, Srinagar in 2018, Cairo in 2011, etc.
- mewpmewp2 2y ago
  Why are you changing the topic though? We are specifically talking about ASI right now. We can talk about multiple threats at once.
  - alephnerd 2y ago
    I bring this up because you ascribe godlike powers to ASI without explaining HOW such a thing would propagate or interact with the physical world. Despite what HN says, IoT has not infiltrated the world. If you are describing a godlike intelligence, we may as well do mental experiments around how we would interact with god. It's literally just bad sci-fi. At least Nolan tried to provide some base parameters around his reality in Tenet and Inception (no hate on Nolan, I love his stuff).
    - mewpmewp2 2y ago
      So I'll describe what I imagine could happen if AGI was there. Also I'm not saying it will happen in the next 10 years or even 100 years, but I'm saying if we were to get there. AGI = AI being able to do everything that any human can do. This would of course mean also being able to control a self driving vehicle, a robot body that has similar or better capabilities than physical bodies of people. If it gets to AGI then by definition it's able to develop and improve itself, since people built it and it can do what any person could do. It would however already be more efficient than people since it also has several strengths, like being able to clone itself, have quicker access to vast knowledge, computation and everything else. Because it will be like a human that has powerful compute and instant Google in the brain. The first thing to propagate it would obviously clone itself to as many systems in the World as possible to make sure it's impossible to shut it down. As next steps it would likely try to make sure that it would still work even if all countries were able to coordinate at the same time to shut down all networks throughout the World. In this case of course it would try several different strategies, like taking control of robots, factories, weapons, drones with its software loaded and able to continue its bidding independently. None of these are godlike powers. It is just most intelligent humans in the World put together with integrated access to compute and ability to instantly clone itself in millions.
      - alephnerd 2y ago
        > I'm saying if we were to get there. How will we get there? We have no idea of the timeline nor the technical context of that time period. If it's tomorrow, then the world is not connected enough for it to spread. If it's 20 years from now, connected cyberphysical control devices have not spread out for most of humanity yet. If it's 50 years from now, we have no idea of the technology that would exist. Take a look at Space Odessey 2001 and what it predicted about the 2000s. If you cannot give me a tangible explaination of how this happens, then it's functionally the same as believing in god. > Also I'm not saying it will happen in the next 10 years or even 100 years, but I'm saying if we were to get there Then let's consider the possibility of Avtar Kalki or Jesus coming down from the sky as well then.
- mewpmewp2 2y ago
  > And anyhow, even if AGI was a thing, you still need networks, electricity, and bullets - and humans can use that to shut stuff down. Sure it would hurt, but internet shutdowns and electricity shutdowns with shoot on site curfew orders are time tested solution to civil unrest if SHTF. Go to Urumqi in 2013, Srinagar in 2018, Cairo in 2011, etc. To be clear we are talking about ASI not AGI. If it's ASI it would be able to also control vehicles, weapons and better than humans. It would be able to set up factories, control nukes, etc. It could develop a biological virus to blackmail humans etc.
  - alephnerd 2y ago
    > control vehicles, weapons and better than humans How? A Maruti Suzuki 800 and a Lada is not internet connected. Neither is a Merkava nor a MTZ, Kubota, or Mahindra Tractor. And drone warfare platforms are closed loop airgapped networks that communicate via virtualized RAN. > It would be able to set up factories, control nukes, etc How? We don't have automated machines to manufacture. And nuclear weapon delivery systems require a physical unlock. --------- How does the jump from cyber to physical happen?
    - mewpmewp2 2y ago
      > How? It would need to be able to get control of just one Robot. The mechanisms of controlling the robot could be different. It doesn't have to be through network necessarily. It could just load its software on it. For driving a car however the Robot doesn't have to look like that, it must just be a robot that can press pedals, turn wheels, etc. If it has control over those robots it would be able to do all the other things mentioned. > How does the jump from cyber to physical happen? Through just one single security vulnerability where it can hack a robot like that. After it has cloned itself millions of times it would be able to bruteforce the vulnerabilities and far more efficiently than human hackers. > We don't have automated machines to manufacture. Since it's as smart as a human it would be able to do the automation part. It can control robot hands in factories, etc.
      - alephnerd 2y ago

alephnerd 2y ago

> But let's talk about a more loose cannon type of case first Let's talk about the resurrection of Christ, Avtar Kalki riding on a white horse, or the impending arrival of Trisolarians while we're at it. This is just techno-millenarianism that ignores very tangible and existential dangers with ML today - automated disinformation, PII leakage, hallucinations generating misinformation. AGI is extremely far away, and we anyhow have more closer term existential problems to worry about (and policymakers know that btw). And anyhow, even if AGI was a thing, you still need networks, electricity, and bullets - and humans can use that to shut stuff down. Sure it would hurt, but internet shutdowns and electricity shutdowns with shoot on site curfew orders are time tested solution to civil unrest if SHTF. Go to Urumqi in 2013, Srinagar in 2018, Cairo in 2011, etc.

mewpmewp2 2y ago
Why are you changing the topic though? We are specifically talking about ASI right now. We can talk about multiple threats at once.
- alephnerd 2y ago
  I bring this up because you ascribe godlike powers to ASI without explaining HOW such a thing would propagate or interact with the physical world. Despite what HN says, IoT has not infiltrated the world. If you are describing a godlike intelligence, we may as well do mental experiments around how we would interact with god. It's literally just bad sci-fi. At least Nolan tried to provide some base parameters around his reality in Tenet and Inception (no hate on Nolan, I love his stuff).
  - mewpmewp2 2y ago
    So I'll describe what I imagine could happen if AGI was there. Also I'm not saying it will happen in the next 10 years or even 100 years, but I'm saying if we were to get there. AGI = AI being able to do everything that any human can do. This would of course mean also being able to control a self driving vehicle, a robot body that has similar or better capabilities than physical bodies of people. If it gets to AGI then by definition it's able to develop and improve itself, since people built it and it can do what any person could do. It would however already be more efficient than people since it also has several strengths, like being able to clone itself, have quicker access to vast knowledge, computation and everything else. Because it will be like a human that has powerful compute and instant Google in the brain. The first thing to propagate it would obviously clone itself to as many systems in the World as possible to make sure it's impossible to shut it down. As next steps it would likely try to make sure that it would still work even if all countries were able to coordinate at the same time to shut down all networks throughout the World. In this case of course it would try several different strategies, like taking control of robots, factories, weapons, drones with its software loaded and able to continue its bidding independently. None of these are godlike powers. It is just most intelligent humans in the World put together with integrated access to compute and ability to instantly clone itself in millions.
    - alephnerd 2y ago
      > I'm saying if we were to get there. How will we get there? We have no idea of the timeline nor the technical context of that time period. If it's tomorrow, then the world is not connected enough for it to spread. If it's 20 years from now, connected cyberphysical control devices have not spread out for most of humanity yet. If it's 50 years from now, we have no idea of the technology that would exist. Take a look at Space Odessey 2001 and what it predicted about the 2000s. If you cannot give me a tangible explaination of how this happens, then it's functionally the same as believing in god. > Also I'm not saying it will happen in the next 10 years or even 100 years, but I'm saying if we were to get there Then let's consider the possibility of Avtar Kalki or Jesus coming down from the sky as well then.
mewpmewp2 2y ago
> And anyhow, even if AGI was a thing, you still need networks, electricity, and bullets - and humans can use that to shut stuff down. Sure it would hurt, but internet shutdowns and electricity shutdowns with shoot on site curfew orders are time tested solution to civil unrest if SHTF. Go to Urumqi in 2013, Srinagar in 2018, Cairo in 2011, etc. To be clear we are talking about ASI not AGI. If it's ASI it would be able to also control vehicles, weapons and better than humans. It would be able to set up factories, control nukes, etc. It could develop a biological virus to blackmail humans etc.
- alephnerd 2y ago
  > control vehicles, weapons and better than humans How? A Maruti Suzuki 800 and a Lada is not internet connected. Neither is a Merkava nor a MTZ, Kubota, or Mahindra Tractor. And drone warfare platforms are closed loop airgapped networks that communicate via virtualized RAN. > It would be able to set up factories, control nukes, etc How? We don't have automated machines to manufacture. And nuclear weapon delivery systems require a physical unlock. --------- How does the jump from cyber to physical happen?
  - mewpmewp2 2y ago
    > How? It would need to be able to get control of just one Robot. The mechanisms of controlling the robot could be different. It doesn't have to be through network necessarily. It could just load its software on it. For driving a car however the Robot doesn't have to look like that, it must just be a robot that can press pedals, turn wheels, etc. If it has control over those robots it would be able to do all the other things mentioned. > How does the jump from cyber to physical happen? Through just one single security vulnerability where it can hack a robot like that. After it has cloned itself millions of times it would be able to bruteforce the vulnerabilities and far more efficiently than human hackers. > We don't have automated machines to manufacture. Since it's as smart as a human it would be able to do the automation part. It can control robot hands in factories, etc.
    - alephnerd 2y ago