It would still be very interesting to study judges' consistency and inter-rater reliability. There was someone's blog post that suggested doing empirical studi

It would still be very interesting to study judges' consistency and inter-rater reliability. There was someone's blog post that suggested doing empirical studies of trials by jury to see how often juries tended to agree with each other (with the suggestion that if there's a very high random component, it's harder to think of the juries' decisions as all reflecting a notion of justice). I've now also forgotten who proposed that.