> Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with

> Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it. Two thoughts. First, in the same way that most genetics research is not human cloning, most AI research is not AGI research. The vast majority of research effort (and industrial investment) is focused on specific tasks: vision, language understanding, planning simple robotic motions, etc. This will likely continue to be true for a long time. Just as geneticists can agree not to apply their techniques to editing human genomes, AI researchers can (hopefully) agree not to attempt to combine their techniques into flexible general-purpose agents, at least until the technology exists to do so safely. Second, there are plenty of ways to work on AGI that do not present immediate containment issues. You can design agents on paper, analyze their behavior with mathematical and theoretical tools, implement and test various subsystems, and even implement and run an entire AGI system on a system with a low enough clock rate that its effective intelligence is less than human. Yes, you need to be careful, and hopefully the field can come to a consensus on the things you shouldn't do. That said I think the primary concerns of a lot of people (Bostrom, etc) are less about academic research gone awry than about potential military or industrial projects, where an arms-race dynamic might cause people to ignore basic safeguards and aim directly towards building highly connected agents running on massively powerful hardware.