3 ms·

> Sprinters don’t accelerate to the top by acquiring the perfect pedigree. They do it by making bold career moves over the course of their career that catapult

> Sprinters don’t accelerate to the top by acquiring the perfect pedigree. They do it by making bold career moves over the course of their career that catapult them to the top. This article looks like yet another one that screams for a survivorship bias disclaimer: https://xkcd.com/1827/ https://xkcd.com/1827/

ngc248 5y ago
Haha yeah, the ones how end up on top always think they made bold moves, while the others didn't. It is exactly as you said, survivor bias and nothing else.
- AmericanChopper 5y ago
  Any study of successful people (or really successful anything, according to any criteria you choose for “success”) can be dismissed as a “survivorship bias”. But it’s quite baseless to presume that no insight can be gained by studying them. What type of bias is it when you just invent an explanation for something out of thin air?
  - dredmorbius 5y ago
    The problem is the sampling. If you select a sample of observations matching your criteria, and then do a factors analysis of them, you're effectively committing the Texas Sharpshooter Fallacy (another way of describing survivorship bias). If instead you were to randomly select from a cohort of individuals and track their career trajectories, then you'd be looking at what factors contribute most to success. I suspect the latter might be rather more strongly weighted to the generally-recommended career advice of good school + business degree + management-track positions early in career, though of course, that's precisely the hypothesis that you're trying to establish. Point being that there are a lot more people who graduate from the School of Hard Knocks and advance through smaller firms (of which there are far more than there are top-school graduates). Then you get to address questions such as whether or not school actually is determinative in careers, or is merely selecting for those destined to succeed regardless. (There was an interesting incidental study done based on NYC magnet school examination cutoffs, which are quite arbitrary, and for which there's significant evidence that those at least immediately bordering the selection criteria (and possibly more broadly) have roughly equivalent career / earnings trajectories). Among the few interesting elements in the HBS "study" is the possible role of high-risk gambles in career advancement. That also ultimately points to survivorship bias (it excludes those who took such gambles and lost), though there's the prospect of multiple-iteration games as well. So long as the upside gain (CEO-track) is high, and the downside loss (failed that advancement attempt, but no penalty) is low, there's little harm in making such attempts. If, however, failed attempts are read as a flag against future consideration, the calculus changes remarkably. This is typical of many "up or out" advancement policies.
    - mgh2 5y ago
      From the book promoted in the article: The CEO next door (p. 22) "With a ‘Moneyball’ approach to leadership..." review Sampling/methodology "One of the reasons for this (stereotypical CEO/survivor bias) is that we tend to limit our thinking to the companies and leaders that regularly appear in mass media. This view—typically focused on Fortune 500 companies—is very narrow. It is also very shallow: we know little about these leaders beyond their seemingly perfect public bios. We tend to ignore the vast universe of companies of all sizes. If you broaden the lens beyond the Fortune 500, there are, for example, over two million companies with more than five employees in the United States alone.5 This means over two million CEOs: a broad, rich set of leadership experiences that don’t often get talked about in the press."
    - AmericanChopper 5y ago
      > Among the few interesting elements in the HBS "study" is the possible role of high-risk gambles in career advancement. That also ultimately points to survivorship bias (it excludes those who took such gambles and lost), though there's the prospect of multiple-iteration games as well. This is a fundamental problem with dismissing all accounts of success so broadly. Multiple-iteration is not a possibility, it’s a certainty. If you take risk in your career, and you fail, the only way that failure can prevent you from trying again is if you fail so hard that you actually die. Failing to death is a tremendously uncommon occurrence when it comes to taking to taking risks in your career. The survivorship bias argument really only works if you imagine each person has a single opportunity through their entire life to achieve success. The Texas sharpshooter fallacy isn’t exactly relevant here either, because that is about re-defining the conditions of success after you’ve taken your measurements. The idea of success here is clearly pre-defined.
      - dredmorbius 5y ago
        Fair points, and I was trying to indicate this myself. More broadly though this highlights the importance of thinking through the model and looking closely at methodology. The HBS article itself really doesn't provide sufficient inforation to judge.
Sebb767 5y ago
Well, it promises the fastest way to become a CEO, not safest once.
- rich_sasha 5y ago
  In expectation, it might not even be the fastest route (I suspect it isn’t). Ie on average following this advice might make it longer to reach CEO position (and you might not reach it). It’s a shame, they have the data and could have asked the question.
dredmorbius 5y ago
Ding ding ding!!! Yes, this.

ngc248 5y ago

Haha yeah, the ones how end up on top always think they made bold moves, while the others didn't. It is exactly as you said, survivor bias and nothing else.

AmericanChopper 5y ago
Any study of successful people (or really successful anything, according to any criteria you choose for “success”) can be dismissed as a “survivorship bias”. But it’s quite baseless to presume that no insight can be gained by studying them. What type of bias is it when you just invent an explanation for something out of thin air?
- dredmorbius 5y ago
  The problem is the sampling. If you select a sample of observations matching your criteria, and then do a factors analysis of them, you're effectively committing the Texas Sharpshooter Fallacy (another way of describing survivorship bias). If instead you were to randomly select from a cohort of individuals and track their career trajectories, then you'd be looking at what factors contribute most to success. I suspect the latter might be rather more strongly weighted to the generally-recommended career advice of good school + business degree + management-track positions early in career, though of course, that's precisely the hypothesis that you're trying to establish. Point being that there are a lot more people who graduate from the School of Hard Knocks and advance through smaller firms (of which there are far more than there are top-school graduates). Then you get to address questions such as whether or not school actually is determinative in careers, or is merely selecting for those destined to succeed regardless. (There was an interesting incidental study done based on NYC magnet school examination cutoffs, which are quite arbitrary, and for which there's significant evidence that those at least immediately bordering the selection criteria (and possibly more broadly) have roughly equivalent career / earnings trajectories). Among the few interesting elements in the HBS "study" is the possible role of high-risk gambles in career advancement. That also ultimately points to survivorship bias (it excludes those who took such gambles and lost), though there's the prospect of multiple-iteration games as well. So long as the upside gain (CEO-track) is high, and the downside loss (failed that advancement attempt, but no penalty) is low, there's little harm in making such attempts. If, however, failed attempts are read as a flag against future consideration, the calculus changes remarkably. This is typical of many "up or out" advancement policies.
  - mgh2 5y ago
    From the book promoted in the article: The CEO next door (p. 22) "With a ‘Moneyball’ approach to leadership..." review Sampling/methodology "One of the reasons for this (stereotypical CEO/survivor bias) is that we tend to limit our thinking to the companies and leaders that regularly appear in mass media. This view—typically focused on Fortune 500 companies—is very narrow. It is also very shallow: we know little about these leaders beyond their seemingly perfect public bios. We tend to ignore the vast universe of companies of all sizes. If you broaden the lens beyond the Fortune 500, there are, for example, over two million companies with more than five employees in the United States alone.5 This means over two million CEOs: a broad, rich set of leadership experiences that don’t often get talked about in the press."
  - AmericanChopper 5y ago
    > Among the few interesting elements in the HBS "study" is the possible role of high-risk gambles in career advancement. That also ultimately points to survivorship bias (it excludes those who took such gambles and lost), though there's the prospect of multiple-iteration games as well. This is a fundamental problem with dismissing all accounts of success so broadly. Multiple-iteration is not a possibility, it’s a certainty. If you take risk in your career, and you fail, the only way that failure can prevent you from trying again is if you fail so hard that you actually die. Failing to death is a tremendously uncommon occurrence when it comes to taking to taking risks in your career. The survivorship bias argument really only works if you imagine each person has a single opportunity through their entire life to achieve success. The Texas sharpshooter fallacy isn’t exactly relevant here either, because that is about re-defining the conditions of success after you’ve taken your measurements. The idea of success here is clearly pre-defined.
    - dredmorbius 5y ago
      Fair points, and I was trying to indicate this myself. More broadly though this highlights the importance of thinking through the model and looking closely at methodology. The HBS article itself really doesn't provide sufficient inforation to judge.

Sebb767 5y ago

Well, it promises the fastest way to become a CEO, not safest once.

rich_sasha 5y ago
In expectation, it might not even be the fastest route (I suspect it isn’t). Ie on average following this advice might make it longer to reach CEO position (and you might not reach it). It’s a shame, they have the data and could have asked the question.

dredmorbius 5y ago

Ding ding ding!!! Yes, this.