2 ms·

Interesting. I'd like to know more about how you setup your experiments.

• rfergie 13y ago
  Have chucked the simulation code up on github: https://github.com/richardfergie/split-test-simulator https://github.com/richardfergie/split-test-simulator Current things that I think are wrong with it: * Successive tests should not be independent; picking the wrong option should have repercussions beyond the next test * The distribution of conversion rates from which samples are drawn was picked because it is convenient not because it is correct * Takes no account of resource issues around creating new variations e.g. it is unlikely that people can create a new variation every day but some strategies assume they can. Please let me know your thoughts