3 ms·
My whole point is that losing control (the Terminator scenario) isn't what we should realistically fear. AI completely under control but following the orders of
by modeless 4y ago
My whole point is that losing control (the Terminator scenario) isn't what we should realistically fear. AI completely under control but following the orders of a ruthless authoritarian regime is really, really bad. We don't need to have faith in some hypothetical godlike paperclip maximizer AI to see that.
- int_19h 4y agoLosing control is not necessarily a "Terminator scenario", and it's unfair to look solely at the most extreme case and its probability to dismiss it. A far more likely case that I can see is subtle bias from the AI injected into decision making processes - it wouldn't even need to be directly wired into anything for that, although that helps. This doesn't require the AI to have agency, even. USSR collapsed in part because the system was entirely planning-centric, and so many people just wrote straight-up bullshit in reports, for reasons ranging from trying to avoid punishment for not meeting goals to trying to cover up personal embezzlement and such. An AI doesn't even need a reason to do that - it can just hallucinate.
- dTal 4y ago"Alignment is unsolved for humans, too."
- int_19h 4y agoIndeed, but we have literally tens of thousands of years of experience with behaviorally modern humans, so we can usually spot the signs much better.