3 ms·
There are great insights that can be learned through these kinds of analyses, but they can also be problematical, suffering from a severe selection (“survivorsh
by jacobolus 15y ago
There are great insights that can be learned through these kinds of analyses, but they can also be problematical, suffering from a severe selection (“survivorship”) bias and confusing correlation with causality. To do a scientific study would require picking some attributes thought to lead to “greatness”, rating a random selection of companies along those attributes, and then tracking their outcomes going forward. Feynman had a great essay about these kinds of problems, “Cargo Cult Science”, which you should go read right now if you never have http://www.lhup.edu/~DSIMANEK/cargocul.htm http://www.lhup.edu/~DSIMANEK/cargocul.htm
For example, the companies featured in Jim Collins’s books haven’t fared especially well in the years after those books were written. Cf. http://www.boston.com/bostonglobe/ideas/articles/2009/04/12/luck_inc/?page=full http://www.boston.com/bostonglobe/ideas/articles/2009/04/12/... http://thehaloeffect.typepad.com/authors_blog/2007/02/jim_collinss_fr.html http://thehaloeffect.typepad.com/authors_blog/2007/02/jim_co... http://bobsutton.typepad.com/my_weblog/2009/04/a-wellcrafted-critique-of-business-success-books-and-my-ambivalence-about-good-to-great-.html http://bobsutton.typepad.com/my_weblog/2009/04/a-wellcrafted... http://hbr.org/2009/04/are-great-companies-just-lucky/ar/1 http://hbr.org/2009/04/are-great-companies-just-lucky/ar/1
- jasonshen 15y agoThat's a great thing to note. Correlation definitely does not indicate causation. In a perfect world, we'd have one group of companies employ the Great by Choice practices, and in another we employ a different set of practices. However, that's not really possible. The point about the other companies is less relevant because companies do change and stop doing all the things that make the great. The Good to Great companies had 15+ years of sustained success - doesn't mean they will be amazing forever. Similarly, just because Arnold Schwarzenegger isn't as ripped as he used to be doesn't mean that lifting heavy weights and eating a lot won't make you gain muscle.
- jacobolus 15y agoYeah, but the point is that “success” depends on a huge number of factors, and a post-hoc ethnographic analysis isn’t necessarily going to pick out what the real differences were. It’s quite easy to fool ourselves, as Feynman describes. Even if you had an economy where all companies were managed identically, the sheer number of companies would lead some of them to be successful for 15 years in a row, and others to go out of business right away. Looking at past history and trying to sift through it for patterns is a great way to form hypotheses and questions for further research, but it’s entirely unscientific to claim those patterns are causal without setting up a new experiment. To take your Schwarzenegger example, we can tell that lifting weights and taking steroids builds muscles because we’ve run the experiment where we take two otherwise identical groups of people, and give one steroids, and then observe the difference going forward. If we only look post-hoc, we might come up with entirely irrelevant patterns, for example finding differences in the amount of time successful bodybuilders spend socializing or watching TV or the number of herbal supplements they take or the kinds of houses they live in, &c. when compared to the general public. Anyhow, I’m not arguing that we shouldn’t try to do ethnographic and historical analyses (my father is an anthropologist/historian!), only that we need to be cautious and skeptical of our conclusions.
- jaylevitt 15y agoYes, if you've read "The Halo Effect", which demolished much of "Good to Great", you'll want to wait for whatever book demolishes "Great by Choice" before reading it too. Decade-old business books are SO much more enlightening; you get to see how history really turned out.
- jasonshen 15y agoThe other important thing to note is that: They didn't just study success, they studied success AGAINST mediocrity These findings aren't just what the 10x companies did. It's what the 10x companies did that the mediocre ones - which were very similar in size, product, team, market - did NOT do or vice versa. They also only selected practices that were done for nearly the entire time studied, in the majority of the companies studied, and that look to be reasonable explanations for the different outcomes. We're not just injecting data into an algorithm. And to respond to jaylevitt - thanks for mentioning Halo Effect. I really should find the time to review it as well.
- jacobolus 15y agoYes, but “pick a smallish set of successful companies and then try to find one other vaguely similar company to each for comparison” is not enough of a control to guard against these kinds of biases. I understand that experiment design is really hard when dealing with something as big and complex as public companies, and Collins does much better than most business books (just by trying at all). But this kind of experiment design would never fly in any hard science.
- mathattack 15y agoThou captured much of my reaction. 9 years doesn't make research rigorous if the methodology is flawed, and the end sample sizes are small. Comparison companies are good but don't solve methodology concerns. And I am actually one who lIkes Jim Collins. It's just bad to view it as rigorous.