4 ms·
To clarify, I meant that you coudln't run multiple tests per page _using_ VWO. Of course, it has been done previously. Yes, running multiple tests is risky and
by paraschopra 17y ago
To clarify, I meant that you coudln't run multiple tests per page _using_ VWO. Of course, it has been done previously.
Yes, running multiple tests is risky and we discourage it because even if you gain speed, the interaction effects may dissolve any results you get.
Again, your last point is excellent. Assigning user ids and then choosing variations dependent on it is a poor strategy. In VWO, we assign variations randomly and then remember (via a cookie) which variation did the user see.
- patio11 17y agoYes, running multiple tests is risky and we discourage it because even if you gain speed, the interaction effects may dissolve any results you get. Ben has IIRC previously written, and I agree, that interaction effects are typically minimal in practice unless you have catastrophic implementation choices. Since you're taking care of the implementation and it isn't catastrophic, you can safely ignore them. Don't advise your users against using your software in a manner likely to increase the value they get out of it. Interaction effects are worth an asterix in the documentation, but not much more.
- paraschopra 17y agoYes, you are probably true. But I have seen users permanently implementing a variation on the slightest hint of improvement, so it is better to take a strict stance because they are going to ignore it anyway.