Correct. For the results presented here, we do not use a neural network during the exploration phase of the algorithm. However, we train a neural network durin

Correct. For the results presented here, we do not use a neural network during the exploration phase of the algorithm. However, we train a neural network during the robustification phase, and it is this trained network that actually plays the game in the end. This final policy is much more robust than the deterministic trajectories collected earlier, and it is able to deal with slight changes in the environment. Note, while not tested yet, you can also run this algorithm by immediately training a policy to reach different states. This variant would be slower (as you now need to train a network during the exploration phase), but this variant would be directly applicable to non-deterministic environments.