3 ms·
If they were running many concurrent, overlapping A/B tests, then they didn't necessarily mess up their data. You are likely to get that result, honestly and tr
by MostlyStable 2y ago
If they were running many concurrent, overlapping A/B tests, then they didn't necessarily mess up their data. You are likely to get that result, honestly and truthfully, purely by chance, if you run enough tests.
Unless the "running concurrent tests and not correcting your significance level", is what you meant by messing up their data, in which case yeah.
- jaggederest 2y agoYes precisely, the latter. I know there are techniques to disentangle that kind of thing but neither I nor they had them readily to hand.