3 ms·

Curious about your use case. Is the idea that you wa t to optimize the conversion and then you would remove the experiment code with the winning variant ?. Or

• joseda-hg 3y ago
  There might be a particular situation where B might be more effective than A, and therefore should be kept, if only for that specific situation There might be a cutoff point, where maintaining B would cost more than it's worth, but that's a parameter you will have to determine for each test
• londons_explore 3y ago
  I'd expect to be running tens of experiments at any one time. Some of those experiments might be variations in wording or colorschemes - others might be entirely different signup flows. I'd let the experiment framework decide (ie. optimize) who gets shown what. Over time, the maintenance burden of tens of experiments (and every possible user being in any combination of experiments) would exceed the benefits, so then I'd want to end some experiments, keeping just whatever variant performs best. And I'd be making new experiments with new ideas.