Agree with all of this except there is no evidence that the A/B testing world as a whole knows anything correct about statistics, in the sense of being able to

Agree with all of this except there is no evidence that the A/B testing world as a whole knows anything correct about statistics, in the sense of being able to profit from more sophisticated bets than "this idea seems not too bad". If players bet real money on the probabilities they claimed, they would be bankrupt. Contrast against the auto insurance companies and such who get this stuff right.