As long as you know which items are random, and you can compute the probability of each item (regardless of whether it's uniform random or through your algorith

As long as you know which items are random, and you can compute the probability of each item (regardless of whether it's uniform random or through your algorithm) for each user, the method still applies. While it does seem more palatable to just show a maximum of X% uniformly random items to all users, the computation of p(y|x) can get a bit hairy for any item y, because now it is a product of a uniform distribution and your stochastic ranking algorithm. If you _can_ compute that, you'd be fine.