The key background here that is somewhat brushed over in the post is that the Beta distribution is conjugate to the Binomial - in the sense that if your paramet

The key background here that is somewhat brushed over in the post is that the Beta distribution is conjugate to the Binomial - in the sense that if your parameter of interest theta has a Beta(a, b) prior, and you observe n successes and r failures from a Binomial trial with parameter(n, theta), your posterior is now Beta(a + n, b + r). The 'non-informative' distribution can be interpreted as having no special information about the prior information of theta, so you just take theta uniform over [0, 1]. This is identical to the Beta(1, 1) distribution. Thus, you obtain the update equations @EvanMiller derived, while making it more clear how to generalize to the informative prior case. In general though, I would have thought one would care more about distributional properties of the posterior p_B - p_A, which are more easily computed by MC methods - the closed form solution for P(p_B > p_A) is nice and fast, but it's not like hypothesis testing is in the critical path of a program, so I wouldn't shy away from MC methods here for a more nuanced view of your posteriors.