The question then is: Is the available data more useful than a coin-flip, which would be the alternative method of making a decision. On the other hand, a coi

The question then is: Is the available data more useful than a coin-flip, which would be the alternative method of making a decision. On the other hand, a coin-flip is probably the better tool. If you can't generate enough data for a statistical sample, then you're probably wasting your time creating an alternative version and setting up an A/B test.