I am not the expert and humbled by all the experts in this thread. But I wanted to say that when I looked A/B testing few years ago I started off with this boo

I am not the expert and humbled by all the experts in this thread. But I wanted to say that when I looked A/B testing few years ago I started off with this book. https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programmin... And somehere down the line he will introduce the Beta/Binomial method for A/B testing. In my (again humble) understanding the benefit of doing this in the Bayesian way is both that you can actually get an understandable answer, but also that the answer does not have to be the final result you can continue by adding a loss function for example.