5 ms·
Assuming that the title question is ever relevant, which it probably will not be, what can we do about it? The most popular proposal is to encourage AI researc
by karmacondon 12y ago
Assuming that the title question is ever relevant, which it probably will not be, what can we do about it?
The most popular proposal is to encourage AI researchers to think about "Friendly" AI, to program morality into their potentially self-aware code and to generally take measures to ensure that any intelligence that they create will be benevolent before it's turned on for the first time. The problem there is that professional AI researchers don't have a monopoly on the technology. Given the numbers, it's more likely that someone who isn't a recognized researcher will create a generally intelligent computer program before a major research institution will. And what then? Unless there's some way to compel every hobbyist and grad student to program a certain level of friendliness into their experiments then all of the warnings and precautions will be for naught.
With known existential threats there are warning signs. If someone is developing a deadly virus or building a dirty bomb they'll most likely have to order specific materials and use certain lab equipment. There will be red flags that someone who is looking closely might have a chance to see. But with a computer program? All someone needs is a computer. Maybe dozens of computers, or more likely dozens of cloud servers hosted with several different companies. If we took this threat very very seriously and tasked a government agency with stopping potentially harmful AI, it would still be almost impossible to catch in time. For all of the famous people sounding the alarm, what solutions are available?
The threat isn't very likely, and unstoppable even if turns out to be real. All we can do is focus on the AI problems that we can solve, develop existing technology and try to chill. If there's one thing human intelligence will always be good at, it's hoping for the best.
- rl3 12y agoRetarding the progress of AI research obviously isn't the solution, precisely for the reasons you outlined. Getting there first, and safely, seems to me the most prudent option. The creation of a Manhattan Project-style program to accelerate progress within a restricted, safety-minded environment may be the best way to proceed. The nature of AGI seems such that the first to attain it may very well determine the future. I just hope that when governments come to this realization, they collaborate or otherwise proceed in secret. Nick Bostrom examined race dynamics between competing projects at length in his book, and it is quite ugly. Safety is the first casualty in most scenarios. That aside, I sincerely hope that the first entity which succeeds in attaining AGI utilizes it for the greater benefit of humanity. Anything else would be petty.
- semi-extrinsic 12y agoI think it's quite safe to assume that the first AGIs will require the biggest supercomputers in the world to run. Since these have severly restricted network access it would be very hard for the AGI to escape. If we're afraid of AGIs the system would likely be airgapped on the first runs. And an escaped AGI running in the cloud or on a botnet would be slowed down by a few orders of magnitude due to network latency, making it less ominous. Bottom line, I don't see the hobbyist/grad student as a problem until we've had AGIs for a long time.