5 ms·

Once my boss asked me if it would be possible to put code in our product that detects if it has been pirated, and if so, formats the hard drive. I told him tha

Once my boss asked me if it would be possible to put code in our product that detects if it has been pirated, and if so, formats the hard drive. I told him that it was a very bad idea for a number of reasons. Primarily I didn't want to have that code in my product because eventually it's going to run in the wrong case. If it's not in there, it won't ever run. I am unconvinced of a lot of the fears around super-AI. I can get behind this initiative though. We have already banned some types of horrible weapons, like flamethrowers and chemical weapons. Hopefully we can manage to ban this one as well.

norea-armozel 11y ago
It's mostly that the AI in question will follow a general directive to its end. For example, ensuring the safety of a nation's population may include putting its citizens into extremely hardened bomb shelters and never let them leave. Or worse, annihilate the entire human species to ensure world peace (the absence of humans would produce the same result and would probably be more efficient in terms of execution). It's not that the AI will be super smart, just that the AI will be super dumb. As Aristotle put it, "Law is mind without reason." And for me, logic is just another set of laws without any sort of reason (justification).
- crazypyro 11y ago
  These sorts of scenarios are the most far-fetched to me. The idea that we will accidentally create a situation in which a logic loophole results in the deaths of billions just sounds ridiculous. It's just a doomsday tale that begins and ends in human imagination.