16 ms·

Yes, I see a huge problem. Preventing damage is an order of magnitude more difficult than causing it.

by ericb 3y ago

juve1996 3y ago
For humans, maybe. Not an AI superintelligence.
- sebzim4500 3y ago
  Citation needed
  - juve1996 3y ago
    The entire GP's argument has no citations either and that is the framework we are working under - that superintelligence can do anything you tell it to do. Ask him for his citation, then the rest follows.
- pixl97 3y ago
  This is not how entropy works. The problem with talking about logical physical systems, is you have to understand the gradient against entropy. There are a trillion more ways to kill you then there are to keep you alive. There is only the tiniest sliver of states in which remain human and don't turn to chemical soup or physics. Any AI capable of it's power bill would be able to tell you that today, and that answer isn't going to change as they get better.
  - juve1996 3y ago
    Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?
    - pixl97 3y ago
      No, the virus is far harder to kill than a human. You have to crate a virus killer that also does not also kill the human host. That is astronomically harder than making a virus that kills.
      - juve1996 3y ago
        If a superintelligence is smart enough to create a virus I'm sure it can also create a virophage to counter it. Whether or not the humans have more than a trillion and viruses only 1 million ways to die, will not have any impact. I suspect both have such a high order of magnitude of ways to die that finding a cross over would be trivial for said superintelligence.
    - PoignardAzur 3y ago
      That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and activate, humanity is still dead.
      - juve1996 3y ago
        > That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point. > So maybe the killer AI would need two months.. Yes, maybe it would. Maybe it wouldn't. Look at every single one of your assumptions - every single one is fiction, fabricated to perfectly sell your story. Maybe the defender AI communicates with the killer AI and comes to a compromise? Why not? We're in la-la-land. Any of us can come up with an infinite number of made up scenarios that we can't prove will actually happen. It's just a moral panic, that will be used by people to their benefit.
oh_sigh 3y ago
There are an order of magnitude more people with orders of magnitude more resources who don't want to die than those who do.
- BeFlatXIII 3y ago
  But it just takes one unforeseen covid-ebola variant to slip through the blind spots of protection-oriented people to be ruinous for humanity.

Yes, I see a huge problem. Preventing damage is an order of magnitude more difficult than causing it.

by ericb 3y ago

Yes, I see a huge problem. Preventing damage is an order of magnitude more difficult than causing it.

juve1996 3y ago

For humans, maybe. Not an AI superintelligence.

sebzim4500 3y ago
Citation needed
- juve1996 3y ago
  The entire GP's argument has no citations either and that is the framework we are working under - that superintelligence can do anything you tell it to do. Ask him for his citation, then the rest follows.
pixl97 3y ago
This is not how entropy works. The problem with talking about logical physical systems, is you have to understand the gradient against entropy. There are a trillion more ways to kill you then there are to keep you alive. There is only the tiniest sliver of states in which remain human and don't turn to chemical soup or physics. Any AI capable of it's power bill would be able to tell you that today, and that answer isn't going to change as they get better.
- juve1996 3y ago
  Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?
  - pixl97 3y ago
    No, the virus is far harder to kill than a human. You have to crate a virus killer that also does not also kill the human host. That is astronomically harder than making a virus that kills.
    - juve1996 3y ago
      If a superintelligence is smart enough to create a virus I'm sure it can also create a virophage to counter it. Whether or not the humans have more than a trillion and viruses only 1 million ways to die, will not have any impact. I suspect both have such a high order of magnitude of ways to die that finding a cross over would be trivial for said superintelligence.
  - PoignardAzur 3y ago
    That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and activate, humanity is still dead.
    - juve1996 3y ago
      > That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point. > So maybe the killer AI would need two months.. Yes, maybe it would. Maybe it wouldn't. Look at every single one of your assumptions - every single one is fiction, fabricated to perfectly sell your story. Maybe the defender AI communicates with the killer AI and comes to a compromise? Why not? We're in la-la-land. Any of us can come up with an infinite number of made up scenarios that we can't prove will actually happen. It's just a moral panic, that will be used by people to their benefit.

oh_sigh 3y ago

There are an order of magnitude more people with orders of magnitude more resources who don't want to die than those who do.

BeFlatXIII 3y ago
But it just takes one unforeseen covid-ebola variant to slip through the blind spots of protection-oriented people to be ruinous for humanity.