4 ms·

Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills

by juve1996 3y ago

Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?

pixl97 3y ago
No, the virus is far harder to kill than a human. You have to crate a virus killer that also does not also kill the human host. That is astronomically harder than making a virus that kills.
- juve1996 3y ago
  If a superintelligence is smart enough to create a virus I'm sure it can also create a virophage to counter it. Whether or not the humans have more than a trillion and viruses only 1 million ways to die, will not have any impact. I suspect both have such a high order of magnitude of ways to die that finding a cross over would be trivial for said superintelligence.
PoignardAzur 3y ago
That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and activate, humanity is still dead.
- juve1996 3y ago
  > That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point. > So maybe the killer AI would need two months.. Yes, maybe it would. Maybe it wouldn't. Look at every single one of your assumptions - every single one is fiction, fabricated to perfectly sell your story. Maybe the defender AI communicates with the killer AI and comes to a compromise? Why not? We're in la-la-land. Any of us can come up with an infinite number of made up scenarios that we can't prove will actually happen. It's just a moral panic, that will be used by people to their benefit.
  - pixl97 3y ago
    >Why couldn't it Tank manufacturers want to know your number...
  - PoignardAzur 3y ago
    > Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. I mean... talking about real-world guns, technology already evolved to the point that doesn't really work anymore? Modern body armor isn't about metal of X thickness, it's about material science and spreading out the impacts, and even then it can only sustain a few direct hits. Like, the asymmetry between offense and defense feels pretty obvious here. You can carry a handgun in a small bag. Bulletproof armor is heavy, cumbersome, conspicuous, etc.
    - juve1996 3y ago
      You're overthinking it. If what you're saying is true, assassinations would run rampant. They don't, despite your perceived asymmetry. The AI doesn't have to stop the gun from firing, or even being manufactured. It has to stop the bullet from hitting the intended target. It can do this in a myriad of ways - like, for example, shooting the other first.

pixl97 3y ago

No, the virus is far harder to kill than a human. You have to crate a virus killer that also does not also kill the human host. That is astronomically harder than making a virus that kills.

juve1996 3y ago
If a superintelligence is smart enough to create a virus I'm sure it can also create a virophage to counter it. Whether or not the humans have more than a trillion and viruses only 1 million ways to die, will not have any impact. I suspect both have such a high order of magnitude of ways to die that finding a cross over would be trivial for said superintelligence.

PoignardAzur 3y ago

That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and activate, humanity is still dead.

juve1996 3y ago
> That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point. > So maybe the killer AI would need two months.. Yes, maybe it would. Maybe it wouldn't. Look at every single one of your assumptions - every single one is fiction, fabricated to perfectly sell your story. Maybe the defender AI communicates with the killer AI and comes to a compromise? Why not? We're in la-la-land. Any of us can come up with an infinite number of made up scenarios that we can't prove will actually happen. It's just a moral panic, that will be used by people to their benefit.
- pixl97 3y ago
  >Why couldn't it Tank manufacturers want to know your number...
- PoignardAzur 3y ago
  > Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. I mean... talking about real-world guns, technology already evolved to the point that doesn't really work anymore? Modern body armor isn't about metal of X thickness, it's about material science and spreading out the impacts, and even then it can only sustain a few direct hits. Like, the asymmetry between offense and defense feels pretty obvious here. You can carry a handgun in a small bag. Bulletproof armor is heavy, cumbersome, conspicuous, etc.
  - juve1996 3y ago
    You're overthinking it. If what you're saying is true, assassinations would run rampant. They don't, despite your perceived asymmetry. The AI doesn't have to stop the gun from firing, or even being manufactured. It has to stop the bullet from hitting the intended target. It can do this in a myriad of ways - like, for example, shooting the other first.