3 ms·
Personally, yes that's what I'm expecting. I don't expect AGI at all. I expect very powerful domain-specific AI (e.g. integrating information, battlespace simu
by crazygringo 4y ago
Personally, yes that's what I'm expecting.
I don't expect AGI at all. I expect very powerful domain-specific AI (e.g. integrating information, battlespace simulation, weapons coordination) that is far more effective than human coordination and communication can be, but I expect it to still just be sticking to pattern recognition, simulating scenarios, and picking the most likely scenarios based on relatively simple reward functions, and with constant human oversight (in the same way managers oversee their employees).
Today's chatbots are controlled perfectly in the sense that they're performing exactly how they were designed to. They're predicting a plausible next word, that's it.
If we start seeing rapidly progressing AGI then it's an entirely different conversation. But there's zero indication that's coming even in a couple of decades. It's 100% hypothetical right now.
- om2 4y agoI don't think LLMs are AGI and I'm not sure if it's an architecture that leads to AGI. But "performing exactly how they were designed to" seems wrong in at least two ways: (1) their creators have done tons of RLHF to try to prevent them from saying things that are false, or racist/sexist, or likely to create danger, but it's pretty easy to get them to do that anyway; (2) lots of specific predictions about LLM capabilities are proven false each time a new one comes out, e.g. being able to pass college-level exams in many subjects. These kinds of behaviors were not designed, they are emergent. You could call this "as designed" in the vacuous sense that the design contains these possibilities even if not anticipated. But then "performing as designed" is not the same as "controlled". Similar levels of unexpected behavior in coordinating weapons would be very very bad.
- goatlover 4y agoAnd what if we connect the chatbots up to all sorts of APIs to perform a million different tasks, and embed them in all sorts of products? Then it will become much more than generating some text when prompted. The LLMs will be in continuous use, with various feedbacks in play. For a Chinese military supremacy scenario, it would be hook the AIs up to all sorts of design, manufacturing, hacking and information warfare processes. Likely this would also secretly include biological, chemical and nanotech lab research as well. That sounds a little scary. It could even just be a stealthy/offensive enough missile design that could justify a first strike. And there's no reason other capable governments won't do a similar thing if we're entering a new arms race. Will it matter so much when a democratic nation achieves supremacy, if the AI is busy designing devastating weapons in a semi-automatic, recursive fashion where nobody really understands the full potential? We're fortunate so far that nothing has triggered a nuclear war.