> First, it isn’t even close to the sort of artificial superintelligence that might conceivably pose a threat to humankind. The models underpinning it are slow

> First, it isn’t even close to the sort of artificial superintelligence that might conceivably pose a threat to humankind. The models underpinning it are slow learners that require immense volumes of data to construct anything akin to the versatile concepts humans can concoct from only a few examples. In this sense, it’s not “intelligent”. Yes, AI is different from the human brain. But human experience is also built up from thousands of hours of high-definition video and audio. An infant brain in an adult body would not be able to do anything "from a few examples". And we have far less knowledge than these LLMs do. An LLM trained solely on reasoning rather than predicting text might be much more capable of reasoning. And current AGI designs work this way because that's the easiest way we've found so far, and we've barely explored the space! It's possible that we will soon discover ways for them to learn much more quickly. > Second, many of the more catastrophic AGI scenarios depend on premises I find implausible. For instance, there seems to be a prevailing (but unspoken) assumption that sufficient intelligence amounts to limitless real-world power. If this was true, more scientists would be billionaires. The flaw in this argument is that scientists are not typically motivated by money, they're motivated by the desire to do science themselves, or maybe by the desire to have high status among scientists. Billionaires are motivated by the desire to have lots of money (or lots of things that money can buy). And scientists are not necessarily more intelligent than billionaires. Maybe most of the AGIs we create won't be good at gaining power. All it takes is a few, and every power-seeking human will try to create power-seeking AGIs for themselves. Does anyone think that politicians or dictators will not try to create AGIs that are good at manipulating public opinion? It's crazy that the author thinks these are strong arguments that AGI is safe. I think it's true that this is a case where humanity needs a security mindset, and we don't. If we did, we'd be tackling anyone who even breathed a word about creating AGI without a bulletproof plan for making sure that it is completely aligned with humanity. To quote Robert Miles: > The trouble with thinking "scientifically" about the future is, you'll never find a peer reviewed study saying what it's like, there can only be speculation. So you never have to reject your "null hypothesis", and that can be anything you want because NH isn't well defined here. > You end up saying "there's no evidence for any particular conclusion", but then go on to act as though you're pretty confident in whatever particular conclusion you arbitrarily picked as your "neutral default"