Isn't that Regression to the Mean[1]? You have two tests, A and B. You take the best N samples by score A, then look at the corresponding score B. Regression to

Isn't that Regression to the Mean[1]? You have two tests, A and B. You take the best N samples by score A, then look at the corresponding score B. Regression to the mean tells you to expect scores B to be lower than A. The classical example is taking the same random test twice and seeing a lower result the second time around. In the post there's a correlation between the tests, so the second test will not have a low value in absolute terms, but will be low relative to the first one (negative correlation). Taking test B before or after the filtering shouldn't matter. My intuition is that this sort of selection picks some samples that got "lucky" on test A, and will score closer to the mean on test B, bringing the distribution down and obscuring or reverting any potential correlations. [1]: https://en.wikipedia.org/wiki/Regression_toward_the_mean https://en.wikipedia.org/wiki/Regression_toward_the_mean