Many Pandora’s boxes. They are all open. The explosion of AI and AGI is not just driven by GPUs, TPUs, and LLMs. That focus in this essay is much too narrow. “

Many Pandora’s boxes. They are all open. The explosion of AI and AGI is not just driven by GPUs, TPUs, and LLMs. That focus in this essay is much too narrow. “Alignment” is always going to be an open problem and each culture will define its own correct and contradictory version. I do not want to settle for the US, Chinese, Russian, or Iranian versions of alignment. This essay is a cry of despair that misses key points. The core problem is how an AI, AGI, or super-intelligent system bootstraps itself to the point that it is able to modulate its “own” attentional systems; to decide what is important and what to do next? What has meaning? Where should I go in space and time? Obviously there are many many solutions. Super-intelligence will not converge on one “truth”. These are challenges of purpose that every organism faces—growth, maintenance, reproduction—but AI systems are fortunately not yet at the point of a self-motivated search for preservation or purpose. The algorithms to add purpose to AI systems will not depend in LLMs. They will depend much more on understanding the computational architectures of core biological/material/energy drivers. These biological algorithms are not actually that complicated, they do not depend on language, but they are damn robust to perturbations. Converting them into computational subsystems (societies of many minds) should not be difficult. This is where Hassabis’s and others who understand neuroscience are critical and have key advantages. Shutting down big LLMs for N months or decades will just move research activity into these other more important conceptual AGI choke points, and probably hasten our approach to full AGI; just the opposite of the intent of the essay. However, this could be a good thing if we can quickly imbue AGIs with emotional intelligence and deep respect for cultural and ecological diversity. (I hear your snorts and laughter.) The opposite of hyper intelligent grey goop. How does any organization train a gentle AGI? Like a child but even more carefully. I would read my AGI baby a lot of books by Rawls, Dewey, and Rorty and then for fun: Bear, Stephenson, Rajaniemi, Egan, Pohl, Sterling, Vinge, and even Wolfe. Full embodiment; full emotional learning; acquisition of purpose; learning to cope with multiple AI cultures. Good luck to us all. I have been hoping to live long enough to view this problem from a distance, but here it is in my backyard. “Forty-two,” said Deep Thought, with infinite majesty and calm.