There seems to be some confusion here with "the usual distribution of ability". >"There will be a few stars who clearly should make the team, and many player

There seems to be some confusion here with "the usual distribution of ability". >"There will be a few stars who clearly should make the team, and many players who clearly shouldn't. " Rather, there are a few who clearly should, a few who clearly shouldn't, and many in-between. Moreover, this assumes there is a quantity called "ability" and that the evaluator is accurately measuring it. I note the later downplaying of accuracy, but before that, it says: >"Probably the difference between them will be less than the measurement error." How do you arrive at an estimate of "measurement error" in order to claim that? Did you first measure the evaluator's "measurement ability"? With what? Even if we naively assume that we can order a group by "ability" with regard to start-ups, and that the difference between adjacent slots is small, it does not follow that your evaluation comes anywhere close to the presumed omniscient set-ordering. The data you have is your self-selected subset, and even there you find that choice #30 is actually in the top few, which is a rather large measurement error; you might argue this doesn't matter in aggregate, but it still argues against any notion of small measurement error. If you can be off by almost your entire selection size then who knows what you missed in the numbers up to twice the chosen cut-off? The statement "it matters least to judge accurately in precisely the cases where judgement has the most effect" seems self-contradictory. What I would guess you mean is that if someone is clearly good, they make the cut, and you don't need to scrutinize to find out how good; but it still matters that you put them on the right side of the cut! It matters the most to judge accurately where judgement has the most effect -- given the nature of start-ups, your missed borderline case could be the one that would have been larger than all the rest put together. Remember your perceived borderline is not the same as the actual borderline. See Bessemer's anti-portfolio: http://www.bvp.com/port/anti.asp http://www.bvp.com/port/anti.asp The article seems to focus on accuracy initially, then semi-mea-culpa its way out of it later. Reading hundreds of applications, your eyes glaze over, and you may only notice someone when they happen to push one of your buttons by chance. It's fine to say you think you selected an "optimal subset" of people according to your checklist, biases, or opinions; but don't confuse that with "ability". You have an idea of what you want and pick according to resemblance to it: small teams that remind you of Viaweb's structure. Using the game theory definitions, I don't find it discouraging that you chose an "optimal" set, which is clearly different in this case from "maximal" ;) Just don't confuse maximal individual ability with optimal set selection, which is what you appear to do with your example of selecting players for a national team. Tanenbaum would've failed Torvalds due to his own bias, which is clearly separate from Linus's ability.