2 ms·
Really found this paper interesting and concerning. I don’t work in marketing or run these kinds of studies. I do work at Qualtrics and have experience with A/B
by gdc3000 2y ago
Really found this paper interesting and concerning. I don’t work in marketing or run these kinds of studies. I do work at Qualtrics and have experience with A/B testing, in general. For those who work in this space and can relate to this paper, would it be helpful if Qualtrics developed some kind of audit panel in our product to help surface potential platform bias? For example, sample ratio mismatch or metadata balance checks.
- robertlagrant 2y agoPerhaps something that highlights the limited scope of a test's predictive power? E.g. for a test run on Facebook "This test is likely to be very useful for a another Facebook ad campaign with the same parameters, and at least somewhat useful for a Google ad campaign with equivalent parameters".