4 ms·
The article has both "suppose it is a normal transformative technology" and "suppose it's not". It's a decision tree supporting the argument for a pause on AI
by bryanlarsen 4y ago
The article has both "suppose it is a normal transformative technology" and "suppose it's not".
It's a decision tree supporting the argument for a pause on AI research.
If it's a normal transformative technology, then a pause won't hurt because it's not a race.
If it is not, then if it is non-aligned it has huge potential for catastrophe. Therefore a pause is essential to ensure that we end up with an aligned AI rather than a non-aligned one.
Choice A: pause won't hurt. Choice B: pause prevents destruction of the world.
- crazygringo 4y agoBut it's not just these two options. I'll take Choice C: alignment is a largely hypothetical problem that probably won't be a major issue at all, while an authoritarian China gaining an insurmountable military advantage would be a here-and-now disaster for those of us who value democracy and self-determination globally. So I'm more concerned about a worryingly realistic problem (China gaining military dominance over the West due to winning an AI race) over a totally hypothetical idea that we're going to reach some kind of singularity where non-aligned AI destroys humanity. If we start getting to a point where a non-aligned AI singularity seems to be a year or two away (or even a couple of months) then yes, shut it all down -- the same way you'd shut down a nuke that turned out to be able to obliterate the entire planet. But right now the singularity is entirely in the realm of science fiction and futurist philosophers. So right now, yes AI does look like a race, and it's important that democracy win that race rather than authoritarianism.
- om2 4y agoSo you think AI will both be overwhelmingly powerful, but also very easy to control reliably? It seems the scenario where C applies implies both of those things, because China only gets an insurmountable advantage if they can build powerful AI but also remain in control. Even today's state of the art chatbots seem pretty hard to control, and they are much less powerful than true AGI so I'm kind of skeptical.
- crazygringo 4y agoPersonally, yes that's what I'm expecting. I don't expect AGI at all. I expect very powerful domain-specific AI (e.g. integrating information, battlespace simulation, weapons coordination) that is far more effective than human coordination and communication can be, but I expect it to still just be sticking to pattern recognition, simulating scenarios, and picking the most likely scenarios based on relatively simple reward functions, and with constant human oversight (in the same way managers oversee their employees). Today's chatbots are controlled perfectly in the sense that they're performing exactly how they were designed to. They're predicting a plausible next word, that's it. If we start seeing rapidly progressing AGI then it's an entirely different conversation. But there's zero indication that's coming even in a couple of decades. It's 100% hypothetical right now.
- om2 4y agoI don't think LLMs are AGI and I'm not sure if it's an architecture that leads to AGI. But "performing exactly how they were designed to" seems wrong in at least two ways: (1) their creators have done tons of RLHF to try to prevent them from saying things that are false, or racist/sexist, or likely to create danger, but it's pretty easy to get them to do that anyway; (2) lots of specific predictions about LLM capabilities are proven false each time a new one comes out, e.g. being able to pass college-level exams in many subjects. These kinds of behaviors were not designed, they are emergent. You could call this "as designed" in the vacuous sense that the design contains these possibilities even if not anticipated. But then "performing as designed" is not the same as "controlled". Similar levels of unexpected behavior in coordinating weapons would be very very bad.
- goatlover 4y agoAnd what if we connect the chatbots up to all sorts of APIs to perform a million different tasks, and embed them in all sorts of products? Then it will become much more than generating some text when prompted. The LLMs will be in continuous use, with various feedbacks in play. For a Chinese military supremacy scenario, it would be hook the AIs up to all sorts of design, manufacturing, hacking and information warfare processes. Likely this would also secretly include biological, chemical and nanotech lab research as well. That sounds a little scary. It could even just be a stealthy/offensive enough missile design that could justify a first strike. And there's no reason other capable governments won't do a similar thing if we're entering a new arms race. Will it matter so much when a democratic nation achieves supremacy, if the AI is busy designing devastating weapons in a semi-automatic, recursive fashion where nobody really understands the full potential? We're fortunate so far that nothing has triggered a nuclear war.