3 ms·
Actually, it seems like the people who are most worried about AGI are the ones building it, because they don't want to see it "raised" the wrong way. Here's a
by muzani 4y ago
Actually, it seems like the people who are most worried about AGI are the ones building it, because they don't want to see it "raised" the wrong way.
Here's a blog by Sam Altman, about 10 months before OpenAI was founded: https://blog.samaltman.com/the-software-revolution https://blog.samaltman.com/the-software-revolution
"Two of the biggest risks I see emerging from the software revolution—AI and synthetic biology—may put tremendous capability to cause harm in the hands of small groups, or even individuals. It is probably already possible to design and produce a terrible disease in a small lab; development of an AI that could end human life may only require a few hundred people in an office building anywhere in the world, with no equipment other than laptops.
...
I think the best strategy is to try to legislate sensible safeguards but work very hard to make sure the edge we get from technology on the good side is stronger than the edge that bad actors get. If we can synthesize new diseases, maybe we can synthesize vaccines. If we can make a bad AI, maybe we can make a good AI that stops the bad one."
- giantg2 4y agoI don't really see how AI itself will end human life. It would have to interact with something else to achieve that.
- muzani 4y agoYou just answered it :p Microsoft trained AI on the internet and it became a racist asshole. OpenAI released ChatGPT and people had fun jailbreaking it to get it to do evil things. Someone could just put AI on a weaponless drone and it could start slashing kids at a playground, deliberately or not. I think what Sam Altman wanted to do was to make sure that if someone created an AI library that was as popular as MySQL, it wouldn't easily develop slasher drone behavior.
- giantg2 4y agoEnding a human life is a little different than ending human life. Especially given the implications posed by his inclusion of mentioning pathogens created in a lab. Even if you have a "slasher drone", the threat itself is not really any different from a manually controlled drone where the operator has similar intentions.