4 ms·

The danger the AI alignment folk are afraid of is completely impossible with current tech, but they want to put up barriers because we have no idea what future

by bnralt 4y ago

The danger the AI alignment folk are afraid of is completely impossible with current tech, but they want to put up barriers because we have no idea what future tech might look like and there’s the possibility some future advance could be very dangerous. When anti-GMO or anti-nuclear folk used this same standard to put up barriers to research into nuclear or GMO research, they get lambasted for being anti-science, but the AI alignment folk get a pass for some reason.

ben_w 4y ago
The only reason I have to think it's impossible for current AI to pay someone to help it bootstrap itself into other hardware is because OpenAI researchers tried to get it to do exactly that and reported that it failed. The only reason I'm confident other AI public models won't determine highly potent novel neurotoxins is that the company who made the AI model which did exactly that thing when they flipped a bit from "least dangerous" to "most dangerous" were absolutely terrified and presumably kept enough away from the public domain. The only reason I'm even hopeful that DNA-on-demand companies keep a watch out for known pathogens is the SciFi about such things going wrong might make them at least try to not do that. Unthinkable man made horrors have been with us for an extremely long time; AI isn't new in this regard, but as intelligence is the human superpower, even in the context of AI that have no agency of their own, it can elevate stupid arseholes to the level of dangerous arseholes.
HDThoreaun 4y ago
The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.
- pantalaimon 4y ago
  A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.
  - HDThoreaun 4y ago
    Nuclear meltdowns don’t have the ability to end humanity
    - ben_w 4y ago
      No, but they can really mess up property prices in the area. Lotta people care about that. Also people are really bad with the relative scale of different big things, so "mess up city" and "mess up planet" come across similarly in people's heads. (This is also why people might try to argue against immigration by saying "America is full" — their city is a bigger part of their world view than is, say, rural Montana). (I am not a huge fan of nuclear power, but I'm also not any kind of opponent; for AI, I can see many ways it might go wrong, but I don't know how to guess at the probability-vs-damage distributions of any of them).
  - bnralt 4y ago
    Indeed. No matter the likelihood of these things happen accidentally, we at least have the ability to create a situation with nuclear power or GMOs that would kill large amounts of people in the present if that was our goal. We couldn’t create a killer AGI right now even if we wanted to and put a huge amount of resources into it. Even if we made one, we don’t know it would be any more powerful than a human who’s paralyzed from the neck down. If you use the same assumptions AI alignment folk use for any other tech (“maybe we’ll be able to create a super powerful version of this even though we currently have no clue how to”/“”maybe that hypothetical super powerful version will be able to destroy the world”), they all become extremely dangerous. The alignment crowd usually handles this by only looking at the known issues for most tech today, but then looking at theoretical unknown issues of futuristic tech years from now when it comes to AI.
- cma 4y ago
  Proliferation?

by bnralt 4y ago

ben_w 4y ago

The only reason I have to think it's impossible for current AI to pay someone to help it bootstrap itself into other hardware is because OpenAI researchers tried to get it to do exactly that and reported that it failed. The only reason I'm confident other AI public models won't determine highly potent novel neurotoxins is that the company who made the AI model which did exactly that thing when they flipped a bit from "least dangerous" to "most dangerous" were absolutely terrified and presumably kept enough away from the public domain. The only reason I'm even hopeful that DNA-on-demand companies keep a watch out for known pathogens is the SciFi about such things going wrong might make them at least try to not do that. Unthinkable man made horrors have been with us for an extremely long time; AI isn't new in this regard, but as intelligence is the human superpower, even in the context of AI that have no agency of their own, it can elevate stupid arseholes to the level of dangerous arseholes.

HDThoreaun 4y ago

The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.

pantalaimon 4y ago
A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.
- HDThoreaun 4y ago
  Nuclear meltdowns don’t have the ability to end humanity
  - ben_w 4y ago
    No, but they can really mess up property prices in the area. Lotta people care about that. Also people are really bad with the relative scale of different big things, so "mess up city" and "mess up planet" come across similarly in people's heads. (This is also why people might try to argue against immigration by saying "America is full" — their city is a bigger part of their world view than is, say, rural Montana). (I am not a huge fan of nuclear power, but I'm also not any kind of opponent; for AI, I can see many ways it might go wrong, but I don't know how to guess at the probability-vs-damage distributions of any of them).
- bnralt 4y ago
  Indeed. No matter the likelihood of these things happen accidentally, we at least have the ability to create a situation with nuclear power or GMOs that would kill large amounts of people in the present if that was our goal. We couldn’t create a killer AGI right now even if we wanted to and put a huge amount of resources into it. Even if we made one, we don’t know it would be any more powerful than a human who’s paralyzed from the neck down. If you use the same assumptions AI alignment folk use for any other tech (“maybe we’ll be able to create a super powerful version of this even though we currently have no clue how to”/“”maybe that hypothetical super powerful version will be able to destroy the world”), they all become extremely dangerous. The alignment crowd usually handles this by only looking at the known issues for most tech today, but then looking at theoretical unknown issues of futuristic tech years from now when it comes to AI.
cma 4y ago
Proliferation?