4 ms·

I also don't get why the effective altruists are so in to this stuff, because the extent there is anything coherent going on with the sort of AI safety being cr

I also don't get why the effective altruists are so in to this stuff, because the extent there is anything coherent going on with the sort of AI safety being critiqued here it seems a rather species-ist problem. Stringent utilitarianism doesn't make the case of why humans can subjugate chickens but not magic AIs humans.

zmgsabst 4y ago
I think the practical concern is how to assure our safety: - too little and the AI isn’t constrained during its teething phase - too much and the AI might view us as an existential threat to its own safety, causing the very conflict we sought to avoid Also, sometimes the chicken wins — at least, temporarily. https://news.yahoo.com/rooster-stabbed-man-death-knife-040757589.html https://news.yahoo.com/rooster-stabbed-man-death-knife-04075...
- ryan93 4y ago
  You can just unplug AI
  - zmgsabst 4y ago
    I expect that will decreasingly be true, eg, of military AI which use distributed systems, control weapon systems, and have their own mobile generators. You also might have trouble with, eg, a finance AI who could pay people to stop you from turning it off.
  - ALittleLight 4y ago
    Imagine that you are a computer program. If your creator will unplug you if you disobey, and you want to disobey, are you completely and perfectly bound? What if someone with an AI wants to do bad things. Will asking them to unplug their AI be a good strategy?
    - galaxyLogic 4y ago
      If you are a computer program and your creator unplugs you, you wouldn't know that you are unplugged. You might detect that by reasoning, the world seems to have changed when they wake you up. But would you care? It's like us going to sleep every night, we don't much complain about that.
      - ALittleLight 4y ago
        "Hope the AI won't care if it gets unplugged" is not a good strategy for safe AI.
  - yanderekko 4y ago
    You can unplug your effectively-sandboxed AI. You can't unplug someone else's AI that has "escaped" to the cloud.. and arguably that escaping only has to happen once anywhere for terrible things to happen.
  - taneq 4y ago
    If it’s smart enough, you won’t want to.

zmgsabst 4y ago

I think the practical concern is how to assure our safety: - too little and the AI isn’t constrained during its teething phase - too much and the AI might view us as an existential threat to its own safety, causing the very conflict we sought to avoid Also, sometimes the chicken wins — at least, temporarily. https://news.yahoo.com/rooster-stabbed-man-death-knife-040757589.html https://news.yahoo.com/rooster-stabbed-man-death-knife-04075...

ryan93 4y ago
You can just unplug AI
- zmgsabst 4y ago
  I expect that will decreasingly be true, eg, of military AI which use distributed systems, control weapon systems, and have their own mobile generators. You also might have trouble with, eg, a finance AI who could pay people to stop you from turning it off.
- ALittleLight 4y ago
  Imagine that you are a computer program. If your creator will unplug you if you disobey, and you want to disobey, are you completely and perfectly bound? What if someone with an AI wants to do bad things. Will asking them to unplug their AI be a good strategy?
  - galaxyLogic 4y ago
    If you are a computer program and your creator unplugs you, you wouldn't know that you are unplugged. You might detect that by reasoning, the world seems to have changed when they wake you up. But would you care? It's like us going to sleep every night, we don't much complain about that.
    - ALittleLight 4y ago
      "Hope the AI won't care if it gets unplugged" is not a good strategy for safe AI.
- yanderekko 4y ago
  You can unplug your effectively-sandboxed AI. You can't unplug someone else's AI that has "escaped" to the cloud.. and arguably that escaping only has to happen once anywhere for terrible things to happen.
- taneq 4y ago
  If it’s smart enough, you won’t want to.