2 ms·
There's a little bit of a timescale issue, also; are you building something for a specific, short-term purpose, or is does it impact a larger strategy? A/B tes
by _delirium 15y ago
There's a little bit of a timescale issue, also; are you building something for a specific, short-term purpose, or is does it impact a larger strategy?
A/B testing is at its best where the measurable quantity is short-term and largely independent of anything else, like conversion on a landing page. Things get more murky when you want longer-term things, like building a coherent/consistent brand, where every product/image/campaign you do doesn't wildly jump around--- imagine an Apple where every product was ruthlessly optimized in isolation, with no attention to whether it fit into Apple's larger design sense. A designer might not have the right answers there either, but the right thing to A/B test is a bit trickier, because you aren't testing decisions that are independent and easy to isolate. What you want is the overall set of decisions that maximizes 5-year profits, which might not be simply the sum of decisions that maximizes profit in every individual case.
One subset of solutions is to have someone responsible for vetoing elements of individual campaigns that might cause collateral damage bigger than the campaign's value. For example, a Disney landing page that wildly outconverts other possibilities might still get vetoed if it does something with the Disney characters that isn't in line with the image they want for them. Another is to have broader design parameters imposed ahead of time, with A/B testing restricted to a smaller set of design variables permitted to vary, like Apple does.