4 ms·
But the alternative to decision by metrics is decision by committee. The former may be gamed, but the latter may be corrupt. The inherent human nature that lead
by IIAOPSW 6y ago
But the alternative to decision by metrics is decision by committee. The former may be gamed, but the latter may be corrupt. The inherent human nature that leads to corruption cannot be fixed, but in theory there could be a metric that isn't flawed. Therefore the solution is not to abandon evidence based policy, but to have more and better evidence based policy.
- caddemon 6y agoI've always wondered about using multiple metrics, but instead of combining them prior to selection, bin the spots and for each bin consider a subset of metrics weighted in different ways. For something like a school ranking, instead of giving a number (which is way more precision than I think these systems can actually achieve), the schools could be tiered - with each bin contributing some spots to the first tier, then repeat. At the very least it should diversify how people try to game the system, and then the "winners" of various bins could be studied to compare usage of different metrics in selection. If the specific formula could be kept mostly secret, perhaps switched up every so often, it would hopefully disincentivize gaming at all. Along those lines, committees and metrics could both be used. Give committees control over 10% of the spots or something. Corruption is obviously a downside, but I do think committees also have upsides that are difficult if not impossible to replicate with pure metrics. In doing this committee selections could also be tracked against the various metric ones for longer term outcomes.