I think better way would be to have very crude 3d procedurally generated world and then use style transfer. This way the world wouldn't change with every step.

I think better way would be to have very crude 3d procedurally generated world and then use style transfer. This way the world wouldn't change with every step. The important parts would still be there, for example if you going towards building it would still be there in every frame just getting closer. At worst it's style would change but not the actual building presence or basic shape.