My opinion is that it's best to just keep running the bandit algorithm. There is no need to stop it -- you can add and remove options as it is running. As for

My opinion is that it's best to just keep running the bandit algorithm. There is no need to stop it -- you can add and remove options as it is running. As for the type of algorithm, I've referenced various bits of the literature elsewhere in the comments. I can't give details on our precise approach unfortunately.