My apologies if my reply came off as snarky. But I think if you're interpreting the OP as sci-fi, you're misreading. Gwern intended it as an illustration of a

My apologies if my reply came off as snarky. But I think if you're interpreting the OP as sci-fi, you're misreading. Gwern intended it as an illustration of a plausible path to AGI takeoff, in the not-too-distant future. The claim is that this could actually happen, in our lifetime, and with no new technology (as Gwern says, "It might help to imagine a hard take⁣off sce⁣nario using solely known sorts of NN & scaling ef⁣fects"). One may reasonably disagree with the claim, by presenting arguments for why takeoff might be harder, or why alignment is easier than this scenario illustrates. But "this scenario could happen" is the explicit, concrete claim.