4 ms·

I don't think the comparison is fair because chemical reactions and how they work was a part of the model of the theory that nuclear testing might blow up the w

I don't think the comparison is fair because chemical reactions and how they work was a part of the model of the theory that nuclear testing might blow up the world. We were dealing with real, testable models of the world all the way up to the nuclear tests. This is more like, if I smash two fish together in the Large Hadron Collider there's a chance the universe ends. Which no one would tell you with a straight face was remotely possible. But! It's never been done before, so, possible? We're human beings. One of the things humans do is project. We do it all the time. To the entire animal kingdom, to the gods of our theologies, you get where I'm going with this? Well now our gods have come to visit us and much like the gods of our fantasies, they're proxies, a mirror to the best and worst of ourselves. Because we still largely operate on fear, we have a camp of end-of-the-worlders who believe these gods are ready to judge us guilty and murderbot us all. Imagine if we were a species where fear was not the dominant feeling. We'd in all likelihood imagine transformers were harbingers of something else entirely.

SonOfLilit 3y ago
Intelligent agents that don't share your worldview can be dangerous. We've seen this a hundred times (e.g. Cortez), I don't think it's controversial or "not part of our model". "The LLM research project might lead to intelligent agents that don't share our worldview" should not be too controversial either? You could argue about the odds, but it's very explicitly the stated goal, and they came orders of magnitude than any before them, and most of the founders of the field are doing things like quitting their prestigious jobs or throwing billions of dollars of investor money in their investors' faces, that they have nothing to personally gain from, for the stated reason of trying to prevent this outcome. So we should probably defer to the experts and assume, say, at least 1% that it's a real risk? 10%? It's not fish in the LHC, it's part of one not unreasonable model of what might happen as a result of people spending billions of dollars trying to make it happen. Edit: I see you've made an important edit. Are we still arguing whether it's possible that OpenAI will succeed in creating intelligent autonomous agents, or only about whether we should fear that eventuality?

SonOfLilit 3y ago

Intelligent agents that don't share your worldview can be dangerous. We've seen this a hundred times (e.g. Cortez), I don't think it's controversial or "not part of our model". "The LLM research project might lead to intelligent agents that don't share our worldview" should not be too controversial either? You could argue about the odds, but it's very explicitly the stated goal, and they came orders of magnitude than any before them, and most of the founders of the field are doing things like quitting their prestigious jobs or throwing billions of dollars of investor money in their investors' faces, that they have nothing to personally gain from, for the stated reason of trying to prevent this outcome. So we should probably defer to the experts and assume, say, at least 1% that it's a real risk? 10%? It's not fish in the LHC, it's part of one not unreasonable model of what might happen as a result of people spending billions of dollars trying to make it happen. Edit: I see you've made an important edit. Are we still arguing whether it's possible that OpenAI will succeed in creating intelligent autonomous agents, or only about whether we should fear that eventuality?