We don’t know what’s inevitable in foresight, only in hindsight. Sparks of AGI are sparks, and do not imply an inevitable progression to a particular end state.

We don’t know what’s inevitable in foresight, only in hindsight. Sparks of AGI are sparks, and do not imply an inevitable progression to a particular end state. I don’t see GPT-* replacing us unless they endow it volition. If it requires prompts to get things done, some human has to understand the system enough in order to provide the correct prompt. Writing code is only a small part of the job. Knowing what code to write, what code to refactor, what abstraction to use that fits with your other abstractions — these things are the real job. Giving GPT volition would make it “self-prompting” and maybe allow it to make these leaps. But volition seems qualitatively different to me than what it’s doing now, and that just improving its current capabilities won’t necessarily lead to volition.