My very recent meta-analysis of 115 A/B tests reveals that a large proportion are highly suspect for p-hacking: http://blog.analytics-toolkit.com/2018/analysis-

My very recent meta-analysis of 115 A/B tests reveals that a large proportion are highly suspect for p-hacking: http://blog.analytics-toolkit.com/2018/analysis-of-115-a-b-tests-average-lift-statistical-power/ http://blog.analytics-toolkit.com/2018/analysis-of-115-a-b-t... Going the Bayesian way, as suggested in some comments, is no solution at all, as I am not aware of an accepted Bayesian approach to dealing with the issue: http://blog.analytics-toolkit.com/2017/bayesian-ab-testing-not-immune-to-optional-stopping-issues/ http://blog.analytics-toolkit.com/2017/bayesian-ab-testing-n... (feel free to run sims, if you do not trust the logic ;-)) as well as on a more general level: http://blog.analytics-toolkit.com/2017/5-reasons-bayesian-ab-testing-debunked/ http://blog.analytics-toolkit.com/2017/5-reasons-bayesian-ab...