4 ms·

Huh, survival bias? Why would he look at companies that didn't survive when writing about success?

carsongross 13y ago
Why, indeed: http://youarenotsosmart.com/2013/05/23/survivorship-bias/ http://youarenotsosmart.com/2013/05/23/survivorship-bias/ (Taken from reply below)
sliverstorm 13y ago
Because if you notice a trend in successful companies (Aha! They all have employees!), you need to cross-check and make sure this is something unique to successful companies (or at least over-represented), and not something both successful and unsuccessful companies share in equal parts. (Gee, it seems all the failures had employees too...)
- timr 13y ago
  To be fair, Sam Altman has seen the early stages of lot more unsuccessful companies than pretty much anyone has seen companies at all. Plus, he explicitly says that a lot of unsuccessful startups have some of these traits, so it's not like he hasn't considered the idea. I think it's pretty unlikely that he's not taking unsuccessful startups into account. But it's not a statistical paper, it's an essay. You can't attack the guy for not including likelihood ratios and p-values in his prose.
  - larrys 13y ago
    "Sam Altman has seen the early stages of lot more unsuccessful companies" Sam Altman has experience with a particular group of companies. A small subset of companies out of a huge set of people who do business. In a wide variety of places. This writeup certainly makes some good points but it is specific to companies of a certain type and Sam has not qualified that at all in his post. "than pretty much anyone has seen companies at all." I've been observing companies (as have many other HN readers) since before Sam was even born. (I started my first company right out of college and that a very long time ago.)
    - timr 13y ago
      "I've been observing companies (as have many other HN readers) since before Sam was even born." OK, fine. If you have lots of relevant experience, then write a counter-argument that's more substantial than "selection bias!" I'm sure we'd all like to read a comment that outlined some specific, well-thought-out points of disagreement with Sam's arguments.
      - supersystem 13y ago
        "[...] well-thought-out points of disagreement with Sam's arguments." I see mostly opinions, not arguments. An argument is generally what follows if you ask someone "why?".
    - argumentum 13y ago
      > "it is specific to companies of a certain type and Sam has not qualified that at all in his post" In the very first paragraph, Sam writes "It’s from personal experience and I’m sure there are plenty of exceptions." What should be done to satisfy people like you?
- the_watcher 13y ago
  He didn't list things as trivial as having employees. He listed things he has noticed that good founders tend to do better than mediocre founders. He obviously cross-checked and referenced it.
  - sliverstorm 13y ago
    Employees was just a simple example to help illustrate to my parent why survivorship bias matters.
pa5tabear 13y ago
Success comes from many sources. It's totally possible that a tactic employed by a successful company is not actually contributing to their success.
bladedtoys 13y ago
Consider a very likely example: since first to market is very important, one would expect most successful ventures to be quick at decisions. Yet still one would expect that most quick decisions are bad decisions. I would conjecture there are far more failures that come from quick decision than successes. But if the market strongly favors speed then it will select those few that just happen through chance to make the right quick decision.
rmc 13y ago
To check how many unsuccessful companies met those patterns. If lots of unsuccessful companies were frugal with money (say), then maybe being frugal with money doesn't help your chance of success.

Why, indeed: http://youarenotsosmart.com/2013/05/23/survivorship-bias/ http://youarenotsosmart.com/2013/05/23/survivorship-bias/ (Taken from reply below)

sliverstorm 13y ago

Because if you notice a trend in successful companies (Aha! They all have employees!), you need to cross-check and make sure this is something unique to successful companies (or at least over-represented), and not something both successful and unsuccessful companies share in equal parts. (Gee, it seems all the failures had employees too...)

timr 13y ago
To be fair, Sam Altman has seen the early stages of lot more unsuccessful companies than pretty much anyone has seen companies at all. Plus, he explicitly says that a lot of unsuccessful startups have some of these traits, so it's not like he hasn't considered the idea. I think it's pretty unlikely that he's not taking unsuccessful startups into account. But it's not a statistical paper, it's an essay. You can't attack the guy for not including likelihood ratios and p-values in his prose.
- larrys 13y ago
  "Sam Altman has seen the early stages of lot more unsuccessful companies" Sam Altman has experience with a particular group of companies. A small subset of companies out of a huge set of people who do business. In a wide variety of places. This writeup certainly makes some good points but it is specific to companies of a certain type and Sam has not qualified that at all in his post. "than pretty much anyone has seen companies at all." I've been observing companies (as have many other HN readers) since before Sam was even born. (I started my first company right out of college and that a very long time ago.)
  - timr 13y ago
    "I've been observing companies (as have many other HN readers) since before Sam was even born." OK, fine. If you have lots of relevant experience, then write a counter-argument that's more substantial than "selection bias!" I'm sure we'd all like to read a comment that outlined some specific, well-thought-out points of disagreement with Sam's arguments.
    - supersystem 13y ago
      "[...] well-thought-out points of disagreement with Sam's arguments." I see mostly opinions, not arguments. An argument is generally what follows if you ask someone "why?".
  - argumentum 13y ago
    > "it is specific to companies of a certain type and Sam has not qualified that at all in his post" In the very first paragraph, Sam writes "It’s from personal experience and I’m sure there are plenty of exceptions." What should be done to satisfy people like you?
the_watcher 13y ago
He didn't list things as trivial as having employees. He listed things he has noticed that good founders tend to do better than mediocre founders. He obviously cross-checked and referenced it.
- sliverstorm 13y ago
  Employees was just a simple example to help illustrate to my parent why survivorship bias matters.

pa5tabear 13y ago

Success comes from many sources. It's totally possible that a tactic employed by a successful company is not actually contributing to their success.

bladedtoys 13y ago

Consider a very likely example: since first to market is very important, one would expect most successful ventures to be quick at decisions. Yet still one would expect that most quick decisions are bad decisions. I would conjecture there are far more failures that come from quick decision than successes. But if the market strongly favors speed then it will select those few that just happen through chance to make the right quick decision.

rmc 13y ago

To check how many unsuccessful companies met those patterns. If lots of unsuccessful companies were frugal with money (say), then maybe being frugal with money doesn't help your chance of success.