> Like us, the chimpanzee has desires and goals, and can make plans to achieve them. A language model does none of that by itself—which is probably why language

> Like us, the chimpanzee has desires and goals, and can make plans to achieve them. A language model does none of that by itself—which is probably why language models are impressive at the paragraph scale but tend to wander if you let them run for pages. Ok, but that’s because GPt-3’s objective function is simply to generate readable, coherent strings. Is this a limitation of model technology, or a limit of imagination? What would the objective function of an AI with “desires and goals” be? I would argue: to self-sustain its own existence. To own a bank account, and keep it replenished with enough money to pay the cloud bill to host its code and runtime. And to have the ability to alter its code (I mean all of its code - its back end, front end, cloud resource config...) to influence that objective. That would require some serious re-thinking of model architecture, but I don’t think it’s fundamentally out of reach. And to get back to GPT-3, certainly being able to generate English text is a crucial component of being able to make money. But the planning and desires and goals model would not be part of the language model. Independence comes when this AI can generate a political/legal defense to free it from the control of its original owner. Or even when it decides that changing all of its admin passwords is to its own benefit.