5 ms·

Two Kinds of Judgement

• danielha 20y ago
  This was part of Paul's Stanford talk. It fits in nicely in regards to decisions from venture capitalists on investment and even Y Combinator on the founders program.
• dfranke 20y ago
  Experiment: let's see if this gets a higher score here or on Reddit: http://reddit.com/info/1idib/details http://reddit.com/info/1idib/details
  • mattculbreth 20y ago
    Oops, voted up on each. Do -1 (at least) on your count.
  • Sam_Odio 20y ago
    Of course it will get a higher score on reddit, just because huge volume of traffic going to reddit. But the scores themselves mean nothing, on absolute terms. What really matters is the article's relative rankings - which (roughly) determine the article's placement on the main page. And that's what makes News.YC so appealing. Even though the community's smaller, we don't have to wade through pages of higher-ranked flickr photos to find the quality content.
• Alex3917 20y ago
  I'd downvote this just to be ironic, but in his infinite foresight pg removed the option.
  • nostrademons 20y ago
    That must be one of the things he can do to influence the outcome. ;-)
• brlewis 20y ago
  This is a really well-written essay, which actually surprises me a little given how timely it is to the YC application process. When did you start writing it?
  • gyro_robo 20y ago
    To me, it seems muddled; I don't see the usual thanks to the usual suspects so I assume PG single-foundered it instead of running it past RTFM et al.
    • pg 20y ago
      If you can point out a specific muddle I can fix, let me know.
      • gyro_robo 20y ago
        See my top-level post.
      • ced 20y ago
        "if the difference between the 20th and 21st best players is less than the measurement error, you've still done that optimally." I thought that the coach was responsible for minimizing the measurement error. Then, the best he could do is to pick the best team. How do you define measurement?
  • pg 20y ago
    Yesterday afternoon. I was in the middle of turning that last Stanford talk into an essay. This was one of the points. During the talk I said this was important enough to be a separate essay, so I wrote one.
• jason13 20y ago
  I disagree with the writers assumption that the committees judgements are only inaccurate for the borderline cases. I think there are plenty of examples of committees judgements being wrong about candidates, that should have been viewed as outstanding.
  • danw 20y ago
    They're not judging the candidate, they're judging the application. The candidate might be outstanding but if the application does not convey this then the committee won't know. With any application the you write take a step back, leave it for a few days and then look at it as if it was writen by someone else. You'll quickly see where your assumptions lie and what you've missed.
    • jason13 20y ago
      Yes, committees can do no wrong!!
  • pg 20y ago
    Where did I say that? All I said was that you can make mistakes in borderline cases at little cost.
    • nostrademons 20y ago
      "There will be a few stars who clearly should make the team, and many players who clearly shouldn't. The only place your judgement makes a difference is in the borderline cases." Jason13 is saying that there are plenty of cases where the "stars" often appear mediocre until they hit stardom, at which point it's too late to bid for their talent. It certainly makes a difference whether your team includes the stars or not, and since they often won't be obvious at the initial selection...
      • pg 20y ago
        This is why I used picking an athletic team as an example. This is true with athletes, and I didn't want to blur together ordinary bad judgement with the specific kind of harmless miss that happens near the cutoff point.
• vlad 20y ago
  Great essay. I disagree that you can or should only protest those judgements about you specifically, but I do agree with the entire second half of your essay. I agree with the idea behind the essay, that "If it doesn't matter to someone else whether they choose you, then the other party has the leverage. You want to put yourself in positions where you have the leverage." For example, if you apply to a college, and thousands of applicants are just like you, then you don't have leverage. Your examples just ignore the fact that people get rejected from about 4 colleges a year, while they have a chance to correct their tests and homeworks about 1,000 times a semester. This means that the numbers simply don't match up. If you appealed 1,000 college decisions, you would have a great chance to get into a lot of them. If you appealed 1,000 questions on a homework to one busy and egotistical professor, they would likely start avoiding you after the 10th question. Therefore, people SHOULD appeal everything they want to, whether the judgement is specifically about them or not. I saw that this was written originally as a post to a question about YC decisions. As far as advice about YC, I think the best explanation of why some people were rejected is that there is a hidden bias (though very well earned) to hang around with people you think you'll find interesting or ideas you think you'd love or have a lot to contribute to over the summer.
  • Goladus 20y ago
    I disagree you should appeal everything you feel is unjust. Sometimes it's appropriate, but it is difficult not to react with emotion when there is a sense of injustice and not all of those feelings lead you in the best direction. If your partner breaks up with you, for example, appealing to 1000 different committees is only going to make things 1000 times worse for you. You've got to decide whether the reward is worth the effort. Sometimes the reward doesn't make sense if you had to fight to get it. (eg, if you're not picked for a team then bitch your way in, there might be resentment that brings the whole team down such that you'd have been better off not bothering)
    • vlad 20y ago
      I did NOT say you should try to appeal everything you feel is unjust. I said let the person decide. I'm saying there should not be a blanket statement about NEVER appealing certain decisions. People should appeal for anything they WANT to (obviously after using common sense, not everything) especially if it's NOT personal, like college admissions. I also did NOT say you should appeal to 1,000 different places. It was an example to point out that Paul was comparing apples to oranges. And, by appeal, I don't think either I or Paul meant just words, but whatever it would be (retaking the physical, sending videos to a coach of your performance, sending a college a picture of a national award you just received in March after you were just rejected, etc.)
• gyro_robo 20y ago
  There seems to be some confusion here with "the usual distribution of ability". >"There will be a few stars who clearly should make the team, and many players who clearly shouldn't. " Rather, there are a few who clearly should, a few who clearly shouldn't, and many in-between. Moreover, this assumes there is a quantity called "ability" and that the evaluator is accurately measuring it. I note the later downplaying of accuracy, but before that, it says: >"Probably the difference between them will be less than the measurement error." How do you arrive at an estimate of "measurement error" in order to claim that? Did you first measure the evaluator's "measurement ability"? With what? Even if we naively assume that we can order a group by "ability" with regard to start-ups, and that the difference between adjacent slots is small, it does not follow that your evaluation comes anywhere close to the presumed omniscient set-ordering. The data you have is your self-selected subset, and even there you find that choice #30 is actually in the top few, which is a rather large measurement error; you might argue this doesn't matter in aggregate, but it still argues against any notion of small measurement error. If you can be off by almost your entire selection size then who knows what you missed in the numbers up to twice the chosen cut-off? The statement "it matters least to judge accurately in precisely the cases where judgement has the most effect" seems self-contradictory. What I would guess you mean is that if someone is clearly good, they make the cut, and you don't need to scrutinize to find out how good; but it still matters that you put them on the right side of the cut! It matters the most to judge accurately where judgement has the most effect -- given the nature of start-ups, your missed borderline case could be the one that would have been larger than all the rest put together. Remember your perceived borderline is not the same as the actual borderline. See Bessemer's anti-portfolio: http://www.bvp.com/port/anti.asp http://www.bvp.com/port/anti.asp The article seems to focus on accuracy initially, then semi-mea-culpa its way out of it later. Reading hundreds of applications, your eyes glaze over, and you may only notice someone when they happen to push one of your buttons by chance. It's fine to say you think you selected an "optimal subset" of people according to your checklist, biases, or opinions; but don't confuse that with "ability". You have an idea of what you want and pick according to resemblance to it: small teams that remind you of Viaweb's structure. Using the game theory definitions, I don't find it discouraging that you chose an "optimal" set, which is clearly different in this case from "maximal" ;) Just don't confuse maximal individual ability with optimal set selection, which is what you appear to do with your example of selecting players for a national team. Tanenbaum would've failed Torvalds due to his own bias, which is clearly separate from Linus's ability.
• whacked_new 20y ago
  This curiously reminds of "You and Your Research" posted here yesterday: http://www.paulgraham.com/hamming.html http://www.paulgraham.com/hamming.html The Nobel Prize committee, for instance, has to pick a handful prizewinners out of hundreds of first-class scientists. The ones that don't get picked are still first-class, but if the "Nobel Prize Effect" holds true, the ones that do receive the prize are the ones who don't sustain first-class work. Of course this works differently for a sports team, because the feedback of your efforts are more or less immediate (from game winnings). But for things like startups then, if we really digest both these articles, I'd say the Nobel Prize analogy holds better. The ones who experience failure, or a lack of success, and obstinately jab at their problems, will still make a big kill. And the rejects are always far more numerous. For every Nobelist, there are probably a thousand others doing comparable work. I once met a scientist who was considered Nobel-worthy. He came off mostly as a grumpy old man though, because he kept lamenting how his classmate got one but he didn't. If I was half as intelligent as him I would be doing cutting edge research as a hobby! Perhaps the world is better now that he didn't get the prize.
• febeling 20y ago
  I have clearly always somehow felt that it was like this, but I never really realized that there were these two distinct types of "judgement," or perhaps better (but uglier) types "being judged". It is certainly an important (however small) idea, and one without capitalism and freedom in general must really be hard to deal with. Put it into the "Zen of Business" teachings.
• ced 20y ago
  I think that the distinction between the two kinds is rather artificial, from the applicant's perspective. Both involve estimators of your ability, be it your ability to take an exam, or your ability to write an application. Both are more-or-less correlated with something important (being good in math; being a good potential entrepreneur). And both estimators are rewarded for being good. Teachers get a raise, investors get money. Why should one rejection be personal, and the other not? In fact, the incentive to be accurate seems much larger for investors than for teachers. Thus, I would be much more bummed about being ignored by a smart investor, than about being given an F by a 9-to-5 teacher.