There seems to be a tension between running the test longer and collecting more profits. If you run the test longer you are less likely to pick the bad option,

There seems to be a tension between running the test longer and collecting more profits. If you run the test longer you are less likely to pick the bad option, at the expense of trying the bad option for a longer period of time. Have you thought about a mathematical solution to this problem? Perhaps there is a good way to decide when to stop automatically.