5 ms·
have you seen an conversion differences in your A/B tests?
by troygoode 15y ago
have you seen an conversion differences in your A/B tests?
- weaksauce 15y agoThat's the push part; implying that the tests have been about equal.
- patio11 15y agoYep. Technically speaking, I got three null results: not enough evidence to reject the null hypothesis (that A was about equal to B). Specifically, I tested 1) Paypal + Google Checkout (my previous arrangement) vs. credit card (Stripe) signups alone 2) PP + GC + CC vs. CC 3) PP + GC + CC with in-application upgrade messages pushing insta-gratification CC-based upgrades vs. plain PP + GC + CC with in-application upgrade messages dumping them to the purchasing page like I have done for the last 3 years n.b. These are my results but they may not be your results. It is entirely possible my credit card form design (which I flagrantly stole from Twitter Bootstrap) is just bad, depressing CC signups, or the UX disimprovement crushed the favored-payment-method improvement, or something else entirely. Test test test.
- carbocation 15y agoAre you measuring, for this specific AB test, the number of conversions? Revenue? Profit? (Admittedly they should all trend in the same direction with the exception of profit, which might depend on how well each company's imposed costs work for your situation.)
- patio11 15y agoNumber of sales. BCC (almost) always sells for $29.95, and the difference between the three in processing costs is rounding error.
- mdda 15y agoI didn't understand "a push" at first : It's a blackjack term : "In the case of a tied score, known as 'push' or 'standoff', bets are normally returned without adjustment; however, a blackjack beats any hand which is not a blackjack, even with value 21. Blackjack vs. blackjack is a push."