4 ms·
While I am generally in favor of applying Bayesian approaches, that's overkill for this problem. In their (fictitious) example, the key problem is that they ra
by jefftk 5y ago
While I am generally in favor of applying Bayesian approaches, that's overkill for this problem. In their (fictitious) example, the key problem is that they ran their test for too short a time. They already know that the typical lag from visit to conversion on their site is longer than a week, which means that if they want to learn the effect on conversions a week isn't enough data.
While it is possible to make some progress on this issue with careful math, simply running the test longer is a far more effective and robust approach.
- OJFord 5y agoI'm no statistician, but don't you have the same problem however long you run it? Giving even more time for slow conversions to amass? Also you and GP are calling the example fictitious, but seems to based on 'real traffic logs' via https://dl.acm.org/doi/10.1145/2623330.2623634 https://dl.acm.org/doi/10.1145/2623330.2623634
- xibalba 5y agoWe're taking the author at his word: > "Let us consider the following fictitious example in which Larry the analyst of the internet company Nozama" Nozama is Amazon backwards.