4 ms·

Be careful about A/B tests, from what I understand Amazon relies on the extensively and they have helped it produce its Amazon website. Enough said. If you ha

by Offler 14y ago

Be careful about A/B tests, from what I understand Amazon relies on the extensively and they have helped it produce its Amazon website. Enough said. If you have two choices one bad and another less bad you will chose the less bad, when instead you may just want to step back and ask is either choice the correct one.

endlessvoid94 14y ago
Are you suggesting Amazon's practices have led it astray somehow?
- rhizome 14y ago
  It's just a pointed way of saying that an important part of testing is knowing what to test.
eclipxe 14y ago
This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.
- hmbg 14y ago
  Thank you. I can't count the times I've tried to explain this to people. The purpose of amazon's site is not that visitors think it's pretty, or even easy to use. It's to get you to buy as much as possible. And there's convincing evidence that it's quite successful in fulfilling that purpose. Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?
alexkus 14y ago
Indeed, not stepping back can lead you (via A/B testing) to just find a local maxima in usability/uptake/etc.
gfodor 14y ago
The problem is though that this observation is often used as an excuse to ignore data or avoid doing small iterative experiments. In most cases, this is a mistake. In other words, you could do a lot worse than reproducing Amazon.com.
kyllo 14y ago
I don't particularly like Amazon's site either, I think it's cluttered, but it has improved a bit. It's still not the prettiest thing, but what it does right, though, is it makes it very, very easy to buy things. And that's all it needs to do. But you're right about not always choosing the "less bad" when there are other options you may not be considering.

by Offler 14y ago

endlessvoid94 14y ago

Are you suggesting Amazon's practices have led it astray somehow?

rhizome 14y ago
It's just a pointed way of saying that an important part of testing is knowing what to test.

eclipxe 14y ago

This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.

hmbg 14y ago
Thank you. I can't count the times I've tried to explain this to people. The purpose of amazon's site is not that visitors think it's pretty, or even easy to use. It's to get you to buy as much as possible. And there's convincing evidence that it's quite successful in fulfilling that purpose. Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?

alexkus 14y ago

Indeed, not stepping back can lead you (via A/B testing) to just find a local maxima in usability/uptake/etc.

gfodor 14y ago

The problem is though that this observation is often used as an excuse to ignore data or avoid doing small iterative experiments. In most cases, this is a mistake. In other words, you could do a lot worse than reproducing Amazon.com.

kyllo 14y ago

I don't particularly like Amazon's site either, I think it's cluttered, but it has improved a bit. It's still not the prettiest thing, but what it does right, though, is it makes it very, very easy to buy things. And that's all it needs to do. But you're right about not always choosing the "less bad" when there are other options you may not be considering.