The real question is "why do people think it's a good idea to create a superintelligence of questionable morality and have the only defense against it be a _hum

The real question is "why do people think it's a good idea to create a superintelligence of questionable morality and have the only defense against it be a _human gatekeeper_". And yes, this notion was seriously proposed. Eliezer does not want to put AIs in boxes. He thinks the entire idea is hopeless; _hence the game_.