6 ms·

Believe it or not I like this being the top story. We know there are lots of bugs in our process and we want to hear the bug reports. I went back to look at t

by pg 15y ago

Believe it or not I like this being the top story. We know there are lots of bugs in our process and we want to hear the bug reports. I went back to look at the application. It fell immediately below the threshold for invitation to interviews. I.e. if even one of us had given them a grade one step better, they would have made it to interviews. When we make mistakes, it's usually in this phase. Most of the time when we screw up it's by not inviting a group to interviews, rather than by interviewing them and turning them down. So this startup is exactly where a mistake would be most likely to occur: in the applications that fell just short of the cutoff for interviews. The good news is, we'd already decided to fix this problem. We're going to do 3 interview tracks for s2012, which will let us interview 270 groups instead of the 180 we did last time. If we'd done that last time, we would have interviewed these guys. But last batch was the first time we did parallel interview tracks at all, and we needed to test whether it worked for 2 before we went to 3.

bicknergseng 15y ago
Look at it this way: at least you interview everyone you accept. Most colleges interview only a few prospective students, and that's for a four year program rather than a three month one.
- bicknergseng 15y ago
  I meant this as a compliment for the hard work you guys do evaluating all the potentials and making deliberate, thoroughly evaluated decisions on who you work with. I'm sorry if I didn't convey that properly.
mirsadm 15y ago
Are you guys keeping track of statistics of grades for applications? I am wondering if at some point there is a clear cut off that you can safely say we don't need to interview these groups. If it is a linear/gradual decrease I'd imagine it would be very difficult to find that threshold.
- pg 15y ago
  The probability that we will fund a startup decreases pretty smoothly as you go down the list. It's not asymptotic at the cutoff. We'd probably have to interview half the applicants to get that last one. Which would kill us.
  - pork 15y ago
    Interesting...you guys sound more and more like a hedge fund (and I say that in a positive light).
j45 15y ago
I can't imagine what it's like to go through so many applicants, so I won't. A few thoughts: - It would be interesting to see what made them 181 instead of 179. Would applicants this close to the cutoff get feedback of any kind about how they fared? - Whoever ranks 271 might be missed out too in the future. - Has any thought been given to sharing "you're this close" details with the 50 or 100 applicants that were within the proverbial whisker, or giving them a second review to see if any would move up or down relative to their peers in the same range? 0.02
- pg 15y ago
  The closer a startup gets to the cutoff, the less there is to say about why they fell below it. I talk about that phenomenon here: http://ycombinator.com/whynot.html http://ycombinator.com/whynot.html
yesimahuman 15y ago
Thanks Paul, we have no hard feelings and are really excited to hear we were that close. We just want to encourage other startups not to give up.
- rexreed 15y ago
  If offered a spot now in YC, would you take it? Not going back in time, but actually, now? If so, and if PG has already admitted it was a mistake, what's to stop YC from offering you a spot in their next session and you taking it? Are there no "re-dos" in YC? Just asking.
  - kamaal 15y ago
    I think that decision boils down to what value YC can bring to their company now.
    - webwright 15y ago
      Many of the companies who apply to YC have this much traction... There's certainly a TON of benefit to be had from YC at their current stage.
    - rexreed 15y ago
      That's my question -- would YC make the offer now, and would the company accept it? Basically, could there possibly be a re-do? The answer might be "no" for both, but it might be "yes". Is applying to YC and accepting a one-shot deal for all parties, or can there ever be a chance to get back together? From the responses and posts, it looks like it's a one-shot deal - either they accept you and you accept the offer, or there's no chance to get together again in the future.
AdamFernandez 15y ago
I don't think much explanation is necessary. YC is never going to catch every potentially successful startup. The process can be refined, but you will never recognize all of them. Averages being what they are, I think YC has done very well. The Codiqa team showed tenacity, and when YC didn't work out for them, they found another route. They are a great example for other startups. None of this should put YC in a negative light.
- pg 15y ago
  Never say never. We may always miss a few, but I think we can get better at judging applications.
bambax 15y ago
I don't know if I'm going to be able to express what I feel but I'll try. YC "bugs" are - startups that are accepted and fail - startups that are rejected and fail, but would have thrived had they been accepted (Of course this last kind of bug is extremely difficult to track down.) But if a startup makes a killing after being rejected by YC it means they didn't need you. If you consider this a bug, there's some kind of moral problem because it means YC is trying to siphon every promising team in order to maximize YC's success, independently of who YC is likely to help more. You're not just betting on horses, you're also training them: it seems what you should want to maximize is YC's impact...?
- jwb119 15y ago
  You're considering success or failure as a binary outcome. It's possible that a startup is accepted and wouldn't have failed anyways, but that being in YC makes it a bigger success than it otherwise would have been (through mentoring or other intervention by YC). My understanding is that is the sweetspot for YC anyways - everyone is a winner there.
  - mirsadm 15y ago
    I think a more important aspect is that YC is able to connect with exceptional individuals (and groups). Even if one particular group fails they may add talented individuals to their network and connect them with other more successful startups.
  - bambax 15y ago
    > You're considering success or failure as a binary outcome I think that's how it is, yes, and PG seems to have stated as much; in essence, you either die or don't die; if you don't die, the scale of your success depends on a lot of things that are beyond your control. (Actual quote from PG: "The formula for the success of a startup includes a random multiplier of roughly zero to a thousand." — http://news.ycombinator.com/item?id=3050280 http://news.ycombinator.com/item?id=3050280 ) If one subscribes to this point of view then the purpose of YC is to help startups not die, and the purpose of the YC admission process is to 1. select all startups that have a chance of not dying if admitted (eg, refuse startups that don't stand a chance in hell of not dying) 2. if the sample from #1 is too large, reduce it at the low end first (startups that are not very likely to not die) but maybe also at the high end (startups that seem certain to not die no matter what) Maybe reducing the selection from the high end is just plain stupid / counter productive, but if you don't do that, it's not very clear why one would be in this business in the first place (as opposed to other lucrative businesses).
joedev 15y ago
This post seems like a success story, so far. So how is not interviewing them a bug? If you are seeking startups who need help identifying how to get customers and how to create products people want, then it seems that Codiqa doesn't fit. Codiqa has customers and is making products people want. Said another way, the delta between where Codiqa stands today vs. where they would be had they been interviewed does not seem as large an opportunity as probably the opportunities you made for the companies you did select. Therefore, I'd say there's no bug.
tapertaper 15y ago
Michael Jordan would WANT to have been rejected.
- coryl 15y ago
  What? Nobody wants to be rejected, ever. Everyone wants to win, always.
cinquemb 15y ago
Here's a possible solution for the future, have YC former accepties conduct proxy interviews for applicants where ever they are in the country(world) on their convenience. It could help you out, unless you feel that the people you accept are unable to help you make your decision. Ivy league schools do it with alumni, YC could do it with alumni.
- tibbon 15y ago
  From what I understand, some of that currently does happen- just not on a huge scale. Alexis is in NYC repping YC there for example.
kamaal 15y ago
I don't know what to reply except that the very words 'process', 'interview', 'batch', 'moderation' these all exist to bring sanity and order into any system at the expense of mediocrity. Like someone said "Micheal Jordan would NOT want to be selected by YC". This means people like Mark Zuckerberg, Larry Page, Sergey brin, Bill Gates, Steve Jobs are not likely to come out of YC. Because the very fact that they have to go through layers of interview sessions, and then spend time performing to you, to get your attention like some performance artist performing to the king to get his attention means that is not going to happen. Because people screw up in interviews. And a interview is generally no measure what the true story actually is. The guide to cracking interviews and performance is learning how to game the system. If your process is known to people outside,and they plan enough to cover all the corners and deal their cards well, they can game your interview process. That's how job hoppers who can do nothing hop around in circles of big successful companies. The situation from evaluating a 'good product' changes to evaluating 'how good the interview went'. Nerds, Geeks of the best quality hate all this interviewing, playing-to-the-galleries-business. That is why they wish to build their own firm. Now if you subject them though the same bureaucratic process which they had otherwise go through in a traditional large corporate(Applying, interviewing etc) they are just going to go away, take money from somebody else and build some stuff in the same time. If I would be building a business and need funding from somebody. I would go to them and explain them 'whats in it for them' and take their money. But if have to go through the same large corporate styles process for this, I will be spending my time for something else. Because the very purpose my starting a business is to avoid that corporate styles processes. You have to seriously ask what value you bring, or how different you are from anybody else. If you are just evaluating profit prospects(which I agree, is important), funding, taking profits back. Then really you are just another VC. There is nothing wrong with that. But that is what it boils down to.
amalakhov 15y ago
Are there any differences between the tracks and interviewing teams (beyond purely idiosyncratic individual preferences/quirks)?
zawaideh 15y ago
I LOVE that you are calling them bugs and taking the selection process very seriously. We (Sandglaz) had a similar experience where we went from 4000 to 35000 users after YC rejection. That said, interviews and applications are inherently subjective and based on a snapshot in time; they will always have "bugs". It is a tough thing to do, and I fully appreciate the effort going into it. Best of Luck!

by pg 15y ago

bicknergseng 15y ago

Look at it this way: at least you interview everyone you accept. Most colleges interview only a few prospective students, and that's for a four year program rather than a three month one.

bicknergseng 15y ago
I meant this as a compliment for the hard work you guys do evaluating all the potentials and making deliberate, thoroughly evaluated decisions on who you work with. I'm sorry if I didn't convey that properly.

mirsadm 15y ago

Are you guys keeping track of statistics of grades for applications? I am wondering if at some point there is a clear cut off that you can safely say we don't need to interview these groups. If it is a linear/gradual decrease I'd imagine it would be very difficult to find that threshold.

pg 15y ago
The probability that we will fund a startup decreases pretty smoothly as you go down the list. It's not asymptotic at the cutoff. We'd probably have to interview half the applicants to get that last one. Which would kill us.
- pork 15y ago
  Interesting...you guys sound more and more like a hedge fund (and I say that in a positive light).

j45 15y ago

I can't imagine what it's like to go through so many applicants, so I won't. A few thoughts: - It would be interesting to see what made them 181 instead of 179. Would applicants this close to the cutoff get feedback of any kind about how they fared? - Whoever ranks 271 might be missed out too in the future. - Has any thought been given to sharing "you're this close" details with the 50 or 100 applicants that were within the proverbial whisker, or giving them a second review to see if any would move up or down relative to their peers in the same range? 0.02

pg 15y ago
The closer a startup gets to the cutoff, the less there is to say about why they fell below it. I talk about that phenomenon here: http://ycombinator.com/whynot.html http://ycombinator.com/whynot.html

yesimahuman 15y ago

Thanks Paul, we have no hard feelings and are really excited to hear we were that close. We just want to encourage other startups not to give up.

rexreed 15y ago
If offered a spot now in YC, would you take it? Not going back in time, but actually, now? If so, and if PG has already admitted it was a mistake, what's to stop YC from offering you a spot in their next session and you taking it? Are there no "re-dos" in YC? Just asking.
- kamaal 15y ago
  I think that decision boils down to what value YC can bring to their company now.
  - webwright 15y ago
    Many of the companies who apply to YC have this much traction... There's certainly a TON of benefit to be had from YC at their current stage.
  - rexreed 15y ago
    That's my question -- would YC make the offer now, and would the company accept it? Basically, could there possibly be a re-do? The answer might be "no" for both, but it might be "yes". Is applying to YC and accepting a one-shot deal for all parties, or can there ever be a chance to get back together? From the responses and posts, it looks like it's a one-shot deal - either they accept you and you accept the offer, or there's no chance to get together again in the future.

AdamFernandez 15y ago

I don't think much explanation is necessary. YC is never going to catch every potentially successful startup. The process can be refined, but you will never recognize all of them. Averages being what they are, I think YC has done very well. The Codiqa team showed tenacity, and when YC didn't work out for them, they found another route. They are a great example for other startups. None of this should put YC in a negative light.

pg 15y ago
Never say never. We may always miss a few, but I think we can get better at judging applications.

bambax 15y ago

I don't know if I'm going to be able to express what I feel but I'll try. YC "bugs" are - startups that are accepted and fail - startups that are rejected and fail, but would have thrived had they been accepted (Of course this last kind of bug is extremely difficult to track down.) But if a startup makes a killing after being rejected by YC it means they didn't need you. If you consider this a bug, there's some kind of moral problem because it means YC is trying to siphon every promising team in order to maximize YC's success, independently of who YC is likely to help more. You're not just betting on horses, you're also training them: it seems what you should want to maximize is YC's impact...?

jwb119 15y ago
You're considering success or failure as a binary outcome. It's possible that a startup is accepted and wouldn't have failed anyways, but that being in YC makes it a bigger success than it otherwise would have been (through mentoring or other intervention by YC). My understanding is that is the sweetspot for YC anyways - everyone is a winner there.
- mirsadm 15y ago
  I think a more important aspect is that YC is able to connect with exceptional individuals (and groups). Even if one particular group fails they may add talented individuals to their network and connect them with other more successful startups.
- bambax 15y ago
  > You're considering success or failure as a binary outcome I think that's how it is, yes, and PG seems to have stated as much; in essence, you either die or don't die; if you don't die, the scale of your success depends on a lot of things that are beyond your control. (Actual quote from PG: "The formula for the success of a startup includes a random multiplier of roughly zero to a thousand." — http://news.ycombinator.com/item?id=3050280 http://news.ycombinator.com/item?id=3050280 ) If one subscribes to this point of view then the purpose of YC is to help startups not die, and the purpose of the YC admission process is to 1. select all startups that have a chance of not dying if admitted (eg, refuse startups that don't stand a chance in hell of not dying) 2. if the sample from #1 is too large, reduce it at the low end first (startups that are not very likely to not die) but maybe also at the high end (startups that seem certain to not die no matter what) Maybe reducing the selection from the high end is just plain stupid / counter productive, but if you don't do that, it's not very clear why one would be in this business in the first place (as opposed to other lucrative businesses).

joedev 15y ago

This post seems like a success story, so far. So how is not interviewing them a bug? If you are seeking startups who need help identifying how to get customers and how to create products people want, then it seems that Codiqa doesn't fit. Codiqa has customers and is making products people want. Said another way, the delta between where Codiqa stands today vs. where they would be had they been interviewed does not seem as large an opportunity as probably the opportunities you made for the companies you did select. Therefore, I'd say there's no bug.

tapertaper 15y ago

Michael Jordan would WANT to have been rejected.

coryl 15y ago
What? Nobody wants to be rejected, ever. Everyone wants to win, always.

cinquemb 15y ago

Here's a possible solution for the future, have YC former accepties conduct proxy interviews for applicants where ever they are in the country(world) on their convenience. It could help you out, unless you feel that the people you accept are unable to help you make your decision. Ivy league schools do it with alumni, YC could do it with alumni.

tibbon 15y ago
From what I understand, some of that currently does happen- just not on a huge scale. Alexis is in NYC repping YC there for example.

kamaal 15y ago

I don't know what to reply except that the very words 'process', 'interview', 'batch', 'moderation' these all exist to bring sanity and order into any system at the expense of mediocrity. Like someone said "Micheal Jordan would NOT want to be selected by YC". This means people like Mark Zuckerberg, Larry Page, Sergey brin, Bill Gates, Steve Jobs are not likely to come out of YC. Because the very fact that they have to go through layers of interview sessions, and then spend time performing to you, to get your attention like some performance artist performing to the king to get his attention means that is not going to happen. Because people screw up in interviews. And a interview is generally no measure what the true story actually is. The guide to cracking interviews and performance is learning how to game the system. If your process is known to people outside,and they plan enough to cover all the corners and deal their cards well, they can game your interview process. That's how job hoppers who can do nothing hop around in circles of big successful companies. The situation from evaluating a 'good product' changes to evaluating 'how good the interview went'. Nerds, Geeks of the best quality hate all this interviewing, playing-to-the-galleries-business. That is why they wish to build their own firm. Now if you subject them though the same bureaucratic process which they had otherwise go through in a traditional large corporate(Applying, interviewing etc) they are just going to go away, take money from somebody else and build some stuff in the same time. If I would be building a business and need funding from somebody. I would go to them and explain them 'whats in it for them' and take their money. But if have to go through the same large corporate styles process for this, I will be spending my time for something else. Because the very purpose my starting a business is to avoid that corporate styles processes. You have to seriously ask what value you bring, or how different you are from anybody else. If you are just evaluating profit prospects(which I agree, is important), funding, taking profits back. Then really you are just another VC. There is nothing wrong with that. But that is what it boils down to.

amalakhov 15y ago

Are there any differences between the tracks and interviewing teams (beyond purely idiosyncratic individual preferences/quirks)?

zawaideh 15y ago

I LOVE that you are calling them bugs and taking the selection process very seriously. We (Sandglaz) had a similar experience where we went from 4000 to 35000 users after YC rejection. That said, interviews and applications are inherently subjective and based on a snapshot in time; they will always have "bugs". It is a tough thing to do, and I fully appreciate the effort going into it. Best of Luck!