3 ms·

Not sure I understand. These are the default survival::survfit CIs. I'd be very interested to hear why they're a poor choice in this case (I don't think they ar

Not sure I understand. These are the default survival::survfit CIs. I'd be very interested to hear why they're a poor choice in this case (I don't think they are) https://stat.ethz.ch/R-manual/R-devel/library/survival/html/survfit.formula.html https://stat.ethz.ch/R-manual/R-devel/library/survival/html/... Also, I have to address the claim of small samples. There is no sample size inherently too small. We make estimates of uncertainty given information. We might wish to extract more certainty (statistical efficiency per n), but that's part of the art. Smaller sample sizes are more work for the statistician but not less valid in any way.

gwern 9y ago
At least one issue is that since the survival curves are different for each batch (as established by your final 'Y Combinator conversion rates by vintage' section), your data is not i.i.d, the K-m curves are changing over time, and so the overall survival curve and its accompany CIs isn't really meaningful: it's sort of the weighted average of the nonparametric K-M curves of each batch further weighted by right-censoring of more recent batches... or something like that. You could try instead showing stratified curves by year, or directly modeling how the hazard rates change over real time. 'survival' claims to support the latter by the 'crazy clock' method (but when I tried doing that in https://www.gwern.net/Google%20shutdowns https://www.gwern.net/Google%20shutdowns I never could figure out how it worked). One thing I wonder is whether YC is scraping the bottom of the barrel and this is why the conversion rates get so much worse, or if it has become more difficult in general to be acquired; if the former, perhaps including the size of the batch as a covariate would be a meaningful test? Data quality might also be an issue from the other comments, and you could try auditing it by picking a few supposed non-exits and researching them intensively to estimate how much measurement error you have. It might also be interesting to see if the survival curves look more like type I, II, or III curves or what parametric distribution might fit best - looks like it could just be exponential/memoryless which is interesting.
- RA_Fisher 9y ago
  Right on! Great feedback! I was also thinking about using competing risks to introduce company foldings. Also parametric modeling as you mention. Try out the Gamma and see which of the family it divines.
  - gwern 9y ago
    Yeah, you could do competing risks. I see yclist.com does distinguish between 'dead' and 'exit', although your CSV doesn't. You can use 'comprsk', I remember it as being no harder to use than 'survival' when I tried it out briefly for https://www.gwern.net/DNM%20survival https://www.gwern.net/DNM%20survival . Any competing-risks model going to be very imprecise, though, since if you can't estimate the overall survival curve very precisely, you definitely can't estimate it split into multiple outcomes.

gwern 9y ago

At least one issue is that since the survival curves are different for each batch (as established by your final 'Y Combinator conversion rates by vintage' section), your data is not i.i.d, the K-m curves are changing over time, and so the overall survival curve and its accompany CIs isn't really meaningful: it's sort of the weighted average of the nonparametric K-M curves of each batch further weighted by right-censoring of more recent batches... or something like that. You could try instead showing stratified curves by year, or directly modeling how the hazard rates change over real time. 'survival' claims to support the latter by the 'crazy clock' method (but when I tried doing that in https://www.gwern.net/Google%20shutdowns https://www.gwern.net/Google%20shutdowns I never could figure out how it worked). One thing I wonder is whether YC is scraping the bottom of the barrel and this is why the conversion rates get so much worse, or if it has become more difficult in general to be acquired; if the former, perhaps including the size of the batch as a covariate would be a meaningful test? Data quality might also be an issue from the other comments, and you could try auditing it by picking a few supposed non-exits and researching them intensively to estimate how much measurement error you have. It might also be interesting to see if the survival curves look more like type I, II, or III curves or what parametric distribution might fit best - looks like it could just be exponential/memoryless which is interesting.

RA_Fisher 9y ago
Right on! Great feedback! I was also thinking about using competing risks to introduce company foldings. Also parametric modeling as you mention. Try out the Gamma and see which of the family it divines.
- gwern 9y ago
  Yeah, you could do competing risks. I see yclist.com does distinguish between 'dead' and 'exit', although your CSV doesn't. You can use 'comprsk', I remember it as being no harder to use than 'survival' when I tried it out briefly for https://www.gwern.net/DNM%20survival https://www.gwern.net/DNM%20survival . Any competing-risks model going to be very imprecise, though, since if you can't estimate the overall survival curve very precisely, you definitely can't estimate it split into multiple outcomes.