3 ms·
I would be careful A/B testing at such a small scale. You are going to hit so much noise, it's not usually a great use of time. Just make it feel good and go f
by brianwawok 6y ago
I would be careful A/B testing at such a small scale. You are going to hit so much noise, it's not usually a great use of time. Just make it feel good and go from there.
A/B testing works great at google scale.
- gigamick 6y agoI hear you and sometimes the results weren't conclusive but I did uncover a few gems. particularly around pricing and the hero statement. I would rather do it than not.
- wpietri 6y agoYou did the right thing. You need Google-scale traffic to tease out subtleties, fractional gains. But at your stage you were looking for big signals. And clearly you found them. Congrats!
- panosfilianos 6y agoThis totally depends on the sample size and the impact that you'd like to measure. You can get very actionable results with a 95% confidence and 80% power at a few thousand sessions.