5 ms·

This is a fair question, but like most things with humans, almost impossible to answer in a definitive way. There was once an accidental control group though.

by paul 12y ago

This is a fair question, but like most things with humans, almost impossible to answer in a definitive way. There was once an accidental control group though. Jessica missed about half of the interviews and PG, RTM, and Trevor were left to decide on their own. I would estimate that the error rate without her was about 2x-3x as high. The other thing to remember is that we keep track of the startups that we interview, and for the ones that we accept, we end up working with them for many years. We make a point of studying our mistakes (and successes), so there is a feedback loop, and I believe that we're getting better at it every time. But of course there are about a million other variable in play, so it's very difficult to be certain about anything (which is part of what makes this business so interesting).

negrit 12y ago
How accurate it is with foreigners? The way to communicate from a country to another is really different, same for the culture, ... How do you integrate the 'foreigner' factor in the evaluation?
- paul 12y ago
  That definitely makes it harder, especially since norms for what is or isn't honest or ethical are far from universal (e.g. what kind of copying is ok and what kind is stealing). This is another reason why it's beneficial to have a culturally diverse team.
  - lazaroclapp 12y ago
    I wonder if formal training in, say, anthropology or social psychology, would help as well. Seems unlikely that you can have a dedicated judge of character from every different culture on earth.
    - deleted 12y ago
      [deleted]
n72 12y ago
> This is a fair question, but like most things with humans, almost impossible to answer in a definitive way. Which is precisely why you have to be very, very, very careful. The matter-of-fact way it was phrased in the blog post didn't give me the impression that there was a whole lot of skepticism regarding her abilities. If one does make such a grandiose claim, it seems like a person conscious that it is a grandiose claim would preface it with a whole lot of qualifiers and such. > There was once an accidental control group though. Jessica missed about half of the interviews and PG, RTM, and Trevor were left to decide on their own. I would estimate that the error rate without her was about 2x-3x as high. I'm curious what the sample-size is and what error-rate actually means. I realize with the control group question and this question, I risk coming off as awfully pedantic.
- lazaroclapp 12y ago
  > Which is precisely why you have to be very, very, very careful. The matter-of-fact way it was phrased in the blog post didn't give me the impression that there was a whole lot of skepticism regarding her abilities. Speaking about unconscious bias, although this judgement can be valid, would you have said the same about say, a (male?) engineer claiming that "I can often tell good software architecture from bad". Which is a similarly subjective, heuristic-based imprecise classification problem. Not saying you wouldn't, by the way. But I wonder if for plenty of us there are biases in play here as well, since we are talking about skills usually considered less "hard" scientifically (even though they are just as much 'hard skills' as the above example about software architecture [1]). [1] Unless you are in the 0.00001% of developers who deal with formally specified and verified software.
  - paul 12y ago
    One of the problems we have in the tech world is that there is a lot of respect for the analytical "left brain" skills (which is great), but a great deal of doubt and skepticism directed towards the more subjective "right brain" skills. I think this is why the "neckbeard" crowd has often dismissed Apple's success as being the product of slick marketing. My opinion is that their success has a lot to do with Job's ability to combine both analytical prowess and creative genius. People who try to be entirely analytical about everything are rarely good founders.
davemel37 12y ago
If this was only about judging how people interact with each other (i.e. do they get along) I would agree the control and feedback might be helpful. But having the ability to judge if someone is a good or bad person or has a good or bad character is the part I can't wrap my head around. The inherent bias is overwhelming as this is an entirely subjective barometer) and the ramifications of playing G-d or Santa sends a screaming message that is counter to everything the post wants me to believe about YC Culture. My definition of a good guy is someone who accepts me for who I am and doesn't pass judgement on me. I suspect most peoples judgement of good or bad character is more a relfection of how they see themselves and how the other person fits into that narrative (i.e. people want to be consistent with their beliefs, so the good/bad evaluation is based on how you perceive yourself and if this person fits into that story), versus any real indication if they are good or not. I guess I'm saying that I am very suspect of the character and judgement of someone who thinks they are good at judging peoples character. Edit: Just to be clear, everything I have ever heard about JL has been wonderful...but this comment about being able to judge character really rubs me the wrong way. Edit2: Grammar, and some elaboration in ().

by paul 12y ago

negrit 12y ago

How accurate it is with foreigners? The way to communicate from a country to another is really different, same for the culture, ... How do you integrate the 'foreigner' factor in the evaluation?

paul 12y ago
That definitely makes it harder, especially since norms for what is or isn't honest or ethical are far from universal (e.g. what kind of copying is ok and what kind is stealing). This is another reason why it's beneficial to have a culturally diverse team.
- lazaroclapp 12y ago
  I wonder if formal training in, say, anthropology or social psychology, would help as well. Seems unlikely that you can have a dedicated judge of character from every different culture on earth.
  - deleted 12y ago
    [deleted]

n72 12y ago

> This is a fair question, but like most things with humans, almost impossible to answer in a definitive way. Which is precisely why you have to be very, very, very careful. The matter-of-fact way it was phrased in the blog post didn't give me the impression that there was a whole lot of skepticism regarding her abilities. If one does make such a grandiose claim, it seems like a person conscious that it is a grandiose claim would preface it with a whole lot of qualifiers and such. > There was once an accidental control group though. Jessica missed about half of the interviews and PG, RTM, and Trevor were left to decide on their own. I would estimate that the error rate without her was about 2x-3x as high. I'm curious what the sample-size is and what error-rate actually means. I realize with the control group question and this question, I risk coming off as awfully pedantic.

lazaroclapp 12y ago
> Which is precisely why you have to be very, very, very careful. The matter-of-fact way it was phrased in the blog post didn't give me the impression that there was a whole lot of skepticism regarding her abilities. Speaking about unconscious bias, although this judgement can be valid, would you have said the same about say, a (male?) engineer claiming that "I can often tell good software architecture from bad". Which is a similarly subjective, heuristic-based imprecise classification problem. Not saying you wouldn't, by the way. But I wonder if for plenty of us there are biases in play here as well, since we are talking about skills usually considered less "hard" scientifically (even though they are just as much 'hard skills' as the above example about software architecture [1]). [1] Unless you are in the 0.00001% of developers who deal with formally specified and verified software.
- paul 12y ago
  One of the problems we have in the tech world is that there is a lot of respect for the analytical "left brain" skills (which is great), but a great deal of doubt and skepticism directed towards the more subjective "right brain" skills. I think this is why the "neckbeard" crowd has often dismissed Apple's success as being the product of slick marketing. My opinion is that their success has a lot to do with Job's ability to combine both analytical prowess and creative genius. People who try to be entirely analytical about everything are rarely good founders.

davemel37 12y ago

If this was only about judging how people interact with each other (i.e. do they get along) I would agree the control and feedback might be helpful. But having the ability to judge if someone is a good or bad person or has a good or bad character is the part I can't wrap my head around. The inherent bias is overwhelming as this is an entirely subjective barometer) and the ramifications of playing G-d or Santa sends a screaming message that is counter to everything the post wants me to believe about YC Culture. My definition of a good guy is someone who accepts me for who I am and doesn't pass judgement on me. I suspect most peoples judgement of good or bad character is more a relfection of how they see themselves and how the other person fits into that narrative (i.e. people want to be consistent with their beliefs, so the good/bad evaluation is based on how you perceive yourself and if this person fits into that story), versus any real indication if they are good or not. I guess I'm saying that I am very suspect of the character and judgement of someone who thinks they are good at judging peoples character. Edit: Just to be clear, everything I have ever heard about JL has been wonderful...but this comment about being able to judge character really rubs me the wrong way. Edit2: Grammar, and some elaboration in ().