4 ms·
I know it's not the point of your post, but I think it's worth pointing out that you're misunderstanding randomness (albeit in a very typical way). Although ran
by advael 3y ago
I know it's not the point of your post, but I think it's worth pointing out that you're misunderstanding randomness (albeit in a very typical way). Although randomness is likely eventually (over a lot of instances) going to be the most "fair" way to distribute where your submission is in the order, it does not guarantee that it will always be different, and in fact a "random" algorithm that took previous orderings into account would be provably less random than one that didn't
It's also worth noting that randomization in a context like this is inherently an imperfect solution to a problem that generally can't be solved perfectly. If we find out that weird ordering biases exist, I think randomization is done on the assumption that many we don't know about could also exist, that there's no clear way to mitigate them completely, and then randomizing the order per-instance is just the best we can do to ensure it's fair (Which, again, won't be perfect. Perfect isn't available)
- barfbagginus 3y agoI don't think they're misunderstanding randomness. They are claiming that randomness is not sufficient for fairness. And I agree. Adding some determinacy to the shuffling design would reduce the variance of unfair scores that students receive. Ie, it could reduce the likelihood that a student's scores will be biased by a run of advantaged or disadvantaged scoring orders. It wouldn't be very hard to design such a scheme. It would produce fairer results in the short run. And in the long run, it would converge to the same fair averages as the fully random process. In fact it would converge faster. This short run statistical efficiency matters. Students don't have access to the long run results - since they're evaluated in the short run.