I think the Intrinsic Motivation is pretty important to solve this, and DeepMind has shown it to work well for games, since it makes the reward far less sparse,

I think the Intrinsic Motivation is pretty important to solve this, and DeepMind has shown it to work well for games, since it makes the reward far less sparse, in a way that is very similar to our own way of playing games - which IMO is somewhat important since we are the intended consumers. Maybe making generic novelty metrics will turn out to be harder than anything else anyway, but it has some intuitive appeal. So I guess we're on the same page, we do need to understand the game output, but IMO understanding it well enough to know when things are novel & interesting vs when things are not is probably enough to get quite far.