To poll meaningful opinion in a highly incentivized environment like that, ratings would have to be strictly relative. Not "is person p1 any good at quality q1,

To poll meaningful opinion in a highly incentivized environment like that, ratings would have to be strictly relative. Not "is person p1 any good at quality q1, q2, q3?", but "quality q1, q2, q3, which one of those is p1's strongest, second strongest, third?". A decision maker could then use that data to extrapolate from qualities he can judge (even if it's as stupid as "attention to personal appearance") to those he can't. People who score badly on a visible quality they are obviously good at must be pretty stellar at the invisible ones, those who score much better than expected in a visible quality, well, there might be a reason for that. Still, that system would be sabotaged by favor based raters. They would quickly rank qualities by perceived importance (I know I would) and blindly rate the most important category highest/lowest depending on popularity. Enter sudoku rules: make it a square of persons/qualities and require both rows and columns to be a complete distribution. The challenge of creating a valid matrix should then take care of any remaining game-theoretic influence, leaving only honest opinion and a fair amount of noise (which is a much lesser evil than deliberate manipulation).