There is an entirely theoretical possibility for this to happen in some universe. It doesn't particularly concern me. Its arguing about how many angels can danc

There is an entirely theoretical possibility for this to happen in some universe. It doesn't particularly concern me. Its arguing about how many angels can dance on a pin's head. The messianic prophecy of the imminence of the in-silico birth of the AGI is based on extreme, simplistic mental models: the brain is a computer, everything is input and output, there is some neural net black box "learning" inside etc. The stance reminds me of the arrogant ignorance displayed by Laplace's demon [1]. Laplace postulated that if someone (the demon) knows the precise location and momentum of every atom in the universe, their past and future values for any given time are entailed - they can be calculated from the laws of classical mechanics. Alas, he completely missed quantum mechanics, the very basis of how the universe works. It was discovered a hundred years after he died. In fact Laplace's overconfidence sin is probably orders of magnitude smaller than what the breathless AGI enthusiasts currently commit: Classical mechanics and quantum mechanics share some formal mathematical similarities [2]. It feels very unlikely that anything resembling AGI will share much mathematical form with current machine learning models. The evolutionary, in-principle agnostic and general purpose "wiring" of the human brain, assuming it can abstracted somehow, lives on a different mathematical plane. [1] https://en.wikipedia.org/wiki/Laplace%27s_demon https://en.wikipedia.org/wiki/Laplace%27s_demon [2] In fact the process of conjuring up the formal apparatus required for quantum mechanics is one of the most dramatic examples of the versatile abstraction powers of the human brain.