2 ms·
> The reason it feels intuitively broken is because the conversion rate is so low. But there were about 10k impressions in his test. Can't be 95% confident in
by akiselev 6y ago
> The reason it feels intuitively broken is because the conversion rate is so low. But there were about 10k impressions in his test.
Can't be 95% confident in something if natural variability drowns out the effect. That kind of conversion rate is one bot RNG away from returning to the mean (noise), it just takes some domain knowledge to recognize why p-hacking is an illusion.