4 ms·
That quote was the sketchiest part of the article for me, because the same math was used to justify the subprime mortgage bubble. The implicit assumption is th
by nostrademons 14y ago
That quote was the sketchiest part of the article for me, because the same math was used to justify the subprime mortgage bubble.
The implicit assumption is that the population of startups is uniform across both the set that YC funds and the set that YC does not fund, such that startups in the latter group have the same chance of being a big hit (modulo YC's mentoring, which is accounted for with the "triple a startup's expected value" clause). But the implication of that assumption is that YC is picking startups at random, and that their filtering process is totally useless!
This sort of math comes up all the time whenever there's a screening process. Imagine that you're hiring for a large tech company, you currently hire 1% of applicants, and you find that among the employees hired, there is no correlation between your interview scores and the employee's eventual job performance. Can you conclude that your interviewing is useless? Should you ramp up hires so you get more workforce of equal quality?
Well, maybe. Because there are a bunch of possible hypotheses that could give this result. Perhaps your interview process is designed to weed out false positives more than false negatives, so it's accurate to the 99th percentile, but then gives no discriminatory power. (Many IQ tests are like this; they're highly correlated with life outcomes up until an IQ of about 140, but beyond that they break down entirely and there's often an inverse correlation with income, happiness, etc. past that). Or perhaps your applicant pool is bimodal: 1% come from other employers and are fully qualified, while 99% are the same jobseekers that every other company rejects. Or perhaps your interview process is broken, and you would do better to find a new one. Or perhaps your interview process is okay, but a number of well-qualified applicants are not even applying to your company.
Which of these is correct? You can't know without randomly sampling the population that was rejected and making an estimate of their quality. This is why all decent scientific experiments have a control, and why financial models get backtested on data that was not part of the training set. Even then, there're lots of things that can go wrong in experiment design, and lots of different ways to interpret data that don't necessarily mean "Fund 10x more startups."
- aptwebapps 14y ago"That quote was the sketchiest part of the article for me, because the same math was used to justify the subprime mortgage bubble." Except that YC doesn't do this. They aren't funding 10x more startups and pg says he avoids finding out how many get funded afterwards because it's the wrong thing to optimize for. To put it another way, there are a lot of ways to bring down post-Demoday funding to 30% and most of them are not going to be helpful. The observation just points out that given their high VC funding rate YC is probably not optimizing for the homeruns as well as it should from a financial perspective.
- nostrademons 14y agoPerhaps I'm misunderstanding this, but: "We'd have to be willing to fund 10x more startups than they would." I guess that if currently nearly all of your startups are getting VC funding, there are two ways to end up funding 10x more startups than VCs: 1.) Make the VCs fund 10x fewer startups or 2.) Fund 10x more startups yourself. (Or various combinations, of course.) #1 seemed absurd to me, so I read the thought experiment as suggesting #2. Edit: Hmm, I guess another alternative is to fund startups that "look worse" to VCs, which is basically #1 but doesn't seem so absurd. So perhaps I was misreading the quote.
- aptwebapps 14y agoEither you are or I am. ;) "We'd have to be willing to fund 10x more startups than they would." The contracted 'we would' is key. They WOULD have to be willing to do that IF they were going to pursue that goal. They aren't. He's positing the idea that if they were properly optimized for the black swans then they would see a much lower funding rate. Not that they should aim for a low funding rate for it's own sake. But, as he discusses, actually performing such an optimization is hard, not to mention the fact that making money isn't their only reason for running YC.
- resbear 14y agoI think the difference here is the assumption for failure rather than success for. The subprime bubble existed due to leverage - fancy models said that defaults were unlikely, so rather than extend loans from their own assets, banks decided to double (or triple, or quadruple...) down and lend out multiples of their own total assets. The models were wrong and they blew up. This is quite different from assuming a high failure rate. The marginal cost of having say, 10% more failed startups isn't a big deal when 95% of them are already failing, and the model accounts for that high failure rate. The black swan is not the unexpected failure that wipes you out, but the unexpected success that outweighs all the failures. Obviously, you'd still be screwed if none of the stuff you invest in pays off. But that would be the case even if your model was more pessimistic; if even 5% success was too much to ask for, perhaps the market for VC is busted, and there was no hope for profit anyway. I don't think pg is implying that screening processes are useless. I'm sure many obviously hopeless ideas get filtered out. It's more that, given the current state of knowledge, everything beyond that basic filter is by nature difficult to discern and we may as well assume a random fat tail distribution.
- nostrademons 14y agoThe comparison I'm trying to draw is about why the models were wrong. Subprime lenders planned for failure too; after all, this is why the loans were subprime, it was expected that a number of them would default. All of this is built into the business model: charge high interest rates on all the loans, to subsidize the cost of some expected number of failures. The problem is that when the business grew and everyone entered, it changed the assumptions that the models were based upon. A certain percentage of mortgages would blow up when subprimes were 1% of the market. The fatal mistake was assuming that the same percentage of mortgages would blow up when subprimes were 10% of the market, because the process of going for 1% to 10% means writing many more loans and extending credit to buyers who should never have been buying houses in the first place.
- anamax 14y ago> The fatal mistake was assuming that the same percentage of mortgages would blow up when subprimes were 10% of the market, because the process of going for 1% to 10% means writing many more loans and extending credit to buyers who should never have been buying houses in the first place. There's another problem - the "market share" of subprime loans was underreported by Fannie and Freddie (and they were the largest single buyer of the relevant securities). So, even if your model of forclosures depended on the share of subprime mortgages and was perfect, you got the wrong results because the inputs were wrong.
- lutorm 14y agoThat quote was the sketchiest part of the article for me, because the same math was used to justify the subprime mortgage bubble. Though PG didn't mention it, from the title it's clear he's alluding to Taleb, who would decidedly not justify the mortgage bubble. The idea is not simply to expose yourself to risk but to make sure you are exposed to large positive outliers while making sure you are not exposed to the negative ones. As resbear said, when you leverage yourself you are increasing your exposure to both positive and negative fluctuations. The long-term success of such a scheme relies on a) accurately estimating the probability of positive vs negative, and b) being able to take the negative ones without blowing up. Neither of those conditions were fulfilled.