But that's not how you should measure it. You goal is not to minimize MSE. Your goal is to rank movies in a way that users like. So the test would be to random

But that's not how you should measure it. You goal is not to minimize MSE. Your goal is to rank movies in a way that users like. So the test would be to randomly split users into test and control, show ranking based on Bayesian averaging to control, show ranking based on James-Stein or some other method to test, measure some metric of user happiness (a different hard problem, click rate on top titles?), then do the comparison.