> Hypothesis 1: Human-level intelligence is possible because brains are biological machines. > Hypothesis 2: A computer with human-level learning abilities wou

> Hypothesis 1: Human-level intelligence is possible because brains are biological machines. > Hypothesis 2: A computer with human-level learning abilities would generally surpass human intelligence because of additional technological advantages. Sure.. but these are sci-fi scenarios. Chatgpt has gotten people thinking, which is good, but let's not lose perspective on what is actually happening before we get too worried. > An AI system in one computer can potentially replicate itself on an arbitrarily large number of other computers to which it has access and, thanks to high-bandwidth communication systems and digital computing and storage, it can benefit from and aggregate the acquired experience of all its clones; this would accelerate the rate at which AI systems could become more intelligent (acquire more understanding and skills) compared with humans. Research on federated learning [1] and distributing training of deep networks [2] shows that this works (and is in fact already used to help train very large neural networks on parallel processing hardware). Now this is speculative and, to be blunt, kind of stupid. The proposed AI is now a self-replicating swarm intelligence that can self-improve and the rate of improvement increases the more it replicates. How'd we get there?!? To work even in a novel you'd going to have to at least hand-wave in some justification for all that. [1] and [2] do not show that scaling compute and storage has no limits in terms of scaling AI intelligence. Seems obvious that there would likely be plateaus. E.g., training an AI on the entire content on the internet probably provides a huge gain over training on a tenth. Training an AI on the entire content of the internet a second time provides no additional gain. Also, it's somehow assumed in here that the suggested future super-intelligent AI will actually have the capability to cause catastrophic harm. I don't get where that assumption comes from. We'd have to grant such an AI some set of magical capabilities, which aren't named. Is "superintelligence" a magical ability to do anything? IDK... to me this reads as "The sky is falling!"