4 ms·

Unless a startup is doing something absolutely terrible, a 45% increase is just not likely. Your advice could better be rephrased as "if you are seeing low enou

by dsl 13y ago

Unless a startup is doing something absolutely terrible, a 45% increase is just not likely. Your advice could better be rephrased as "if you are seeing low enough numbers that it effects the viability of your company, redesign X and then A/B test the change." Startups have a hundred better things they could be spending time on. A/B testing preys on a founders sense that they are leaving money on the table and not optimizing for maximum profit, sometimes to the detriment of focusing on things like good hires or critical product changes. Edit: It isn't obvious until I clicked around, but the author is a little biased and should disclose he is the "Head of Optimization" at Optimizely

colemorrison 13y ago
Ahh... That last part makes sense. This is what I was trying to get around to. If you only have a handful of designers and developers, you shouldn't be focusing on min/maxing your landing page. It's not that this is a "myth", it just doesn't make sense for a new team to worry about it vs product development. I'm not claiming this to be truth or anything. Just in my experience. Bringing up a/b testing to my clients or teams just results in a "we have better things to do." (I suppose being the head at a co. that does this would make it biased on at least some level).
- kylerush 13y ago
  This is a completely fair point. Startups naturally will have to decide where the highest ROI is. It may not be a/b testing. However, they should know that a/b testing isn't necessarily off the table.
alanfalcon 13y ago
It's pretty easy for absolutely terrible decisions to be made at a smaller startup, especially when you don't have the tools handy to prove that they're terrible decisions. I happen to work at a smallish eCommerce startup with lower traffic volumes and a 5% (checkout–not engagement) conversion rate, and while A/B testing small (or even large) changes to our homepage haven't proven fruitful, two small changes to our product pages have made a meaningful difference to our bottom line. (And I mean small as in I spent maybe an hour total setting up those tests while on an Optimizely free plan.) So yeah, I'm an Optimizely and A/B test fan.

by dsl 13y ago

colemorrison 13y ago

Ahh... That last part makes sense. This is what I was trying to get around to. If you only have a handful of designers and developers, you shouldn't be focusing on min/maxing your landing page. It's not that this is a "myth", it just doesn't make sense for a new team to worry about it vs product development. I'm not claiming this to be truth or anything. Just in my experience. Bringing up a/b testing to my clients or teams just results in a "we have better things to do." (I suppose being the head at a co. that does this would make it biased on at least some level).

kylerush 13y ago
This is a completely fair point. Startups naturally will have to decide where the highest ROI is. It may not be a/b testing. However, they should know that a/b testing isn't necessarily off the table.

alanfalcon 13y ago

It's pretty easy for absolutely terrible decisions to be made at a smaller startup, especially when you don't have the tools handy to prove that they're terrible decisions. I happen to work at a smallish eCommerce startup with lower traffic volumes and a 5% (checkout–not engagement) conversion rate, and while A/B testing small (or even large) changes to our homepage haven't proven fruitful, two small changes to our product pages have made a meaningful difference to our bottom line. (And I mean small as in I spent maybe an hour total setting up those tests while on an Optimizely free plan.) So yeah, I'm an Optimizely and A/B test fan.