> we trust that the positive effects, on balance, will outweigh the negative ones. So far, the positive effects from technology definitely seem like they've ou

> we trust that the positive effects, on balance, will outweigh the negative ones. So far, the positive effects from technology definitely seem like they've outweighed the negative ones. Technology has raised our standards of living, and we've managed to work around many potential downsides. Most technologies to date have a fairly limited per-use radius of impact. Guns for example, are local: bullets don't travel more than a couple miles. Mishaps with guns (e.g. shooting sprees) can be terrible, but they have a small radius of impact, so they're not the end of the world. As technologies become more powerful, it's reasonable to assume that maximum radius of impact will also increase. If a radius of impact is the size of the earth (or larger), we have to worry very much about accidents, even if we can expect accidents to be rare. AI might be one of the first technologies humanity comes up with that has a global per-use effect radius. Extreme caution is warranted.