4 ms·
This problem is already solved with humans, some of whom are also very intelligent and goal oriented. It's done by noticing when those rare individuals do thing
by lopmotr 7y ago
This problem is already solved with humans, some of whom are also very intelligent and goal oriented. It's done by noticing when those rare individuals do things we don't want and formalizing our preference in law. So many people agree on the concept of respecting the law that we all cooperate to enforce it. The law is an ugly mess of rules made to plug gaps in itself as they were taken advantage of. Nobody could have worked it all out up front and programmed it into a robot. It evolved along with the intelligent humans it was meant to control. It's precarious though - if the majority of humans don't keep vigilantly maintaining it, an intelligent human might move too fast and gain enough power to take control of the updating of the law! Then we have something like a tyranny. Which can and does sometimes kill all the humans (within its influence) in pursuit of its goals. So as long as we don't let robots move too fast for us to notice what's happening, we should be able to just keep writing more and more laws for them to follow. Not moving too fast for us to keep an eye on them might be the first one!
- throwaway2048 7y agoThe problem is "move slow" is itself a rule that is subject to violation, and "move fast" might be so fast that its nearly instantaneous in its catastrophe (say an AI that nukes the entire world)