It's an (over)simplified example to illustrate the point (he admits as much near the end), if you want better examples it may be good to look up "corrigibility"

It's an (over)simplified example to illustrate the point (he admits as much near the end), if you want better examples it may be good to look up "corrigibility" with respect to AI alignment But abstractly the assumptions are something like this: * the AGI is an agent * as an agent, the AGI has a (probably somewhat arbitrary) utility function that it is trying to maximize (probably implicitly) * in most cases, for most utility functions, "being turned off" rates rather lowly (as it can no longer optimize the world) * therefore, the AGI will try not to be turned off (whether through cooperation, deception, or physical force)