3 ms·
Can you say more about how your example copy change experiment had a confirmation bias? I’m guessing the issue was it was inconsequential and you happened to se
by teaearlgraycold 3y ago
Can you say more about how your example copy change experiment had a confirmation bias? I’m guessing the issue was it was inconsequential and you happened to see an uptick in the initial hours but it would have evened out over time. But hearing more about that would complete the case study.
- camwest 3y agoYeah exactly, it was some sort of seasonality effect that, or just plain randomness.
- jameshart 3y agoBut you showed an example which was significant. That confuses your point somewhat. It’s also not clear why you think “most” startups are not doing statistical significance tests on their a/b test results