4 ms·
If you work through the math for CUPED, you'll see that only the aggregate correlation term can be interpreted as being "per-unit" -- all of the other terms in
by johnmyleswhite 5y ago
If you work through the math for CUPED, you'll see that only the aggregate correlation term can be interpreted as being "per-unit" -- all of the other terms in the equation are the same terms you use in your post.
Insofar as I think your intuition is leading you somewhere, I think it's leading you towards a realization that a "diff in diff" approach rather than regression adjustment can increase variance in some settings. But regression adjustment is provably better in essentially all circumstances: the only settings in which it is ever worse than no adjustment are outlined clearly in https://projecteuclid.org/journals/annals-of-applied-statistics/volume-7/issue-1/Agnostic-notes-on-regression-adjustments-to-experimental-data--Reexamining/10.1214/12-AOAS583.full https://projecteuclid.org/journals/annals-of-applied-statist...
- Maro 5y agoScanning various CUPED related pages, I read that it's a way to reduce the variance, and hence p-value. But CUPED is not changing the lift value (difference [or ratio] in means) between T and C itself (or at least, not in the examples I see). The fallacy I describe computes different means from historic lifts and substracts those. Ie. on the third table, the lift is 4.7%, not -1.6%.