14 ms·

Why Big Companies Keep Failing: The Stack Fallacy

408 points by walterclifford 11y ago

ogezi 11y ago
A company should always focus on your strengths if not it'll be both overstretched and unsuccessful. Great read.
- justicezyx 11y ago
  I do not think that's the correct conclusion. The conclusion should be that when venture into new areas, be sure to understand the market first.
  - 2np 11y ago
    Or, just because you understand the underlying technology doesn't mean you understand the market, and your previous experience will get in the way of actually understanding the market. If you want to go down the tech stack, though, you might be successful because you're part of the market for that tech.
    - anshublog 11y ago
      Exactly. When you go down the stack like Amazon did with AWS, you know your customer (often you). You also know the key pain points and gaps in the market. This is especially so in "as a service" markets.
    - anshublog 11y ago
      Exactly. When you go down the stack like Amazon did with AWS, you know your customer (often you). You also know the key pain points and gaps in the market. This is especially so in "as a service" markets.
adevine 11y ago
I don't buy the arguments in this article. For example, the whole part about why Google failed with Google+ is just wrong IMO. It wasn't that Google wasn't capable of building a good social network. If anything, I (and most people I know) preferred the design and interface of Google+. The problem was that Facebook already had a huge head start, and all your friends were already there. Facebook was "good enough", and there wasn't a big enough incentive to want to switch to Google+. If anything, large companies often miss out on new trends and changes in business and technology, but it's not solely because building that one new layer "up the stack" is so technically hard or different.
- riyadparvez 11y ago
  Also there is this thing called 'network effect' which plays a huge role for social networks. EDIT: Why the downvote?
  - danieltillett 11y ago
    I didn't down vote you, but my guess it is the tone of your comment. You are of course 100% right that for a general social network there can only be one. Until Facebook messes up it won't be possible to build a new general social network - maybe News Corp will buy Facebook and an opening will arise.
- rajangdavis 11y ago
  I think this example adds truth to the author's argument in that there have been a few social media apps like Instagram and Snapchat that took off in ways the Google Plus never did. I do agree that there was little impetus to move to Google Plus from Facebook, but there could have been features that would have made it enough of a differentiator that people could have moved over to Google Plus. It's not like Google lacked the ability to scale out or provide such a service.
  - adevine 11y ago
    I would argue that Snapchat and Instagram occupy VERY different niches than Facebook. Most importantly, virtually everyone who uses Snapchat or Instagram also uses FB. Google thought they had enough clout to win the "general all purpose category" social network - they were trying to compete directly against FB.
kazinator 11y ago
This stack fallacy sounds very familiar. Oh, we will just have a few system calls like open, close, read and write, some TTY and credentials related stuff, a bit of signal handling, process control with fork, exect and wait ... writing a shell language on top will practically just be a footnote.
- anshublog 11y ago
  Yes many open source projects suffer from this too. We as engineers (even former ones) often assume the what to build is "obvious". This is often not true, if ever. The best open source projects are ones where engineers are the end users (customers).
  - marshray 11y ago
    Perhaps this is a feeling that, as experienced implementers, we have a big head start on the hard parts. Maybe this is another form of the "solution looking for a problem" phenomenon.
anjc 11y ago
I'd like some solid examples of what companies confirmed perceptions of competitors were in the same vertical, versus the reality. Because even the author references competency-based views of competitive advantage, but for some reason ignores resource based views, and ignores the fact that companies might be aware of their competences. That is to say, I'm sure that large companies tend to mostly be aware of what their competences are based on the resources and knowledge that they have. If they don't have marketing departments that have analyzed the ERP market, sales teams with ERP training, tech departments with key HR, key knowledge etc etc, then I'm certain they are very well aware of this. Maybe some companies have had marketing missteps and have made poor strategic and competitive decisions, however, but I really doubt that it's due to a lack of introspection or simple analysis as described. Also, IBM didn't "think nothing much" of the software layer. They misunderstood the nature of power in the supply chain, and most importantly, didn't solidify their position within the supply chain while they were dominant.
- anshublog 11y ago
  They think they know their competencies (database systems in one case, virtualization in another) but often they misunderstand what customers really value up the stack - in this example, Workday customers really don't care if it runs Oracle or MySQL or non RDBMS at all. In AWS case, we don't care for kvm vs VMware wars. (I am the author of the TechCrunch post.)
gregdoesit 11y ago
From my experience the same (often random) reason that makes a company succeeds, then becomes their DNA, and finally can make them fail. I saw this happen with Skype where I worked a couple of years. The company succeeded because of P2P: we grew with little infrastructure to reach 200M+ people. P2P became our DNA, rooted deep within (almost) every core component. Then came the new wave of mobile messaging apps. We reacted... with a P2P messaging solution. It was obvious this wasn't working - you sent a message to someone from Skype for iPhone, and they got it... sometime. We knew to have a chance against Whatsapp and other messaging apps we needed server based messaging, so we built it. It took 3 years. Yes, it took this long to get rid of the P2P code from just the messaging components from the 20+ Skype products - we had 1,000+ engineers and 50+ internal teams by the end which significantly slowed things down. When we were done and popped the champagne - no one really cared. And yes, the source code is still full of P2P references and workarounds to this date.
- im2w1l 11y ago
  Why was it hard to deliver messages to iphone using p2p? I mean I guess they couldn't open ports and listen for messages, but that should have been true for plenty of NATed systems too right?
  - joshyeager 11y ago
    We used Skype IM for a long time (inertia). It wasn't just the iPhone. Skype message delivery was erratic on all platforms. It got better, but even now there are occasional delivery delays when I chat with someone who hasn't switched over (we use Slack now).
    - gregdoesit 11y ago
      Yes, there are still traces of the p2p code. Most of messaging has been migrated to server based back ends, but there are still some hybrid solutions in place. Funny thing the company DNA, P2P is still haunting Skype.
      - 11y ago
mwnz 11y ago
Do big companies really keep failing? I'm failing to see the evidence of that assertion.
- sawyer 11y ago
  Failing to gain traction upstream, not failing to keep the lights on.
libertymcateer 11y ago
Apple is not vertically integrated - Wikipedia entries to the contrary notwithstanding. It is a grossly inaccurate statement. Up until very recently, Apple didn't own a single factory - how can one possibly claim that they are vertically integrated if they don't own their own means of production? Apple is a fantastically successful software and industrial design company. The vast majority of their production is outsourced. This is not vertical integration. Additionally, actually, Apple has tremendous amounts of hugely successful and popular software. Though I dig the underlying point of this article, that product management is hard, I think the examples are less than good.
- vacri 11y ago
  Where do you draw the line? If a company does own the factory that builds their widgets, do you claim they're not vertically integrated because they don't own the mines that get the materials, nor the transport network to get it to and from the factory? At the other end, do they need to own the bank with which they take payments from customers? You don't need to own all elements in a vertical stack to be vertically integrated.
  - josu 11y ago
    Apple sells iphones. Who makes the iphones? Not Apple.
    - beambot 11y ago
      Apple designs iPhones. They manage the fabrication of parts very closely and manage their assembly and QA more closely than most of their competitors (like, big enough to make demands of Foxconn or cause them to build specialized factories). Then they sell the phones directly to consumers. They are (literally) on the Wikipedia page as an example of a vertically integrated company: https://en.wikipedia.org/wiki/Vertical_integration https://en.wikipedia.org/wiki/Vertical_integration I'm not saying you're wrong... but it doesn't look like you're right.
    - vacri 11y ago
      You're playing fast and loose with the definition of 'make' there. 'Physically assemble' is not the only element of 'make'.
libertymcateer 11y ago
Apple is not vertically integrated - Wikipedia entries to the contrary notwithstanding. It is a grossly inaccurate statement. Up until very recently, Apple didn't own a single factory - how can one possibly claim that they are vertically integrated if they don't own their own means of production? Apple is a fantastically successful software and industrial design company. The vast majority of their production is outsourced. This is not vertical integration. Additionally, actually, Apple has tremendous amounts of hugely successful and popular software. Though I dig the underlying point of this article, that product management is hard, I think the examples are less than good.
marshray 11y ago
> “Can we compete with Intel or SAP?” Well for one thing we know that Intel spends several $billion to open a new semiconductor plant and has a dozen of them already. https://en.wikipedia.org/wiki/List_of_Intel_manufacturing_sites https://en.wikipedia.org/wiki/List_of_Intel_manufacturing_si... Whereas SAP is, well, a lot of software. Which is something, but Intel needs to make a lot of software too, and chip designs are in some ways a specialized form of software. So I think in some sense Intel is strictly more challenging to replicate than SAP. (But this is probably just my misunderestimation talking. :-)
- at5 11y ago
  You're actually quite right. Capital intensiveness is a barrier to entry. That's why software businesses rise rapidly and fall rapidly at a greater frequency than other types of businesses.
anonymousguy 11y ago
The solution to stack fallacy is simple but really counter-intuitive. All of the mentioned examples, I mean every single one, indicate a business trying to force its way into the higher level through business channels. For example, when a business wants into a higher level they make it a business priority to create a new product and attempt to drive the priorities of this next level product through their business objectives. That is an epic fail. It is important to instead concede that you don't know the needs of the consumers in the higher level, and if you think you do it is because you are guessing. The only way avoid the problem is to not attempt to move into the higher level, at least not intentionally and not through business priorities. This is extremely counter-intuitive because there are generally fewer expenses and greater market frequency at each higher level, which means superior revenue potential. Businesses exist to make money and to ignore moving up to the higher level means denying this potential (vast) revenue source. This doesn't mean you can't move into the higher level of the stack and be really good at it. It just means you cannot do so both intentionally and as a business objective. The solution is to double-down on where you already are with what you are already good at and focus on product quality of your existing products. Continue to improve where you are already good. Improvements and enhancements to existing products can gradually yield the next level as the improvements progressively open new potential in an evolutionary fashion. While getting to the next level this way is much slower it is also risk reduced and continues to associate your brand with quality and consume satisfaction. This will only work, though, if the goal is improving the current product and not acquiring revenue in that desired higher level. Think evolution and not revolution. It has to be a gradual, almost accidental, increase of capability based on meeting current consumer needs.
- PhilWright 11y ago
  I don't think this works in practice. I cannot see Oracle slowly adding extra features to its database until it becomes an actual CRM system. A CRM is not just a solution to a different business problem but also a logically different layer from a development perspective. You would have to build it as a product that sits on top of the database product. But as a CRM system it has no value to the end customer unless it has a certain minimum level of functionality. So although you can add features gradually and evolve your CRM offering you will not have a single customer until you have actually added enough features that make it actually usable. Nobody will buy your gradually evolving system until several years have passed and it works for some actual customers. In which case you have already spend alot of money and effort.
tn13 11y ago
There was a brilliant essay by an Indian politician few years back after his party lost the elections. Later in lecture he explain why political parties and large companies have so much in common when it comes to failing. His basic logic was that - Success depends on processes - Processes even though might be thought of as abstract in reality are function of people at top. - Company gets successful because some bright guy is the rebel, he questions status quo, persists and succeeds. - As time goes by, the rebellious ideas actually become conservative ideas. The rebel is now on top. As his ideas fade he struggles to stay on top. - He recruits people who see the world through him, he builds processes that enforce that vision. - This makes it difficult for the truth to be visible to the top management. - By the time failure is visible it is hard to turn around the ship. - IN SHORT: Companies/Nations fail because someone at top did not know when to quit. - In the end that rebel turned conservative becomes bitter. He thinks the world owed him something for what he achieved. He explained who USSR examples. How a genetic scientist got promoted because his fake research re-enforced something that Stalin had said long back and his peers were scared to point out the fact because it might get perceived as anti-Stalin. I observed Blackberry very closely and it resonated to me so much. The founders at one point blamed people for using iphone and not blackberry. Best companies in the world are seem to be those where their top leaders quit at their peak to make way for their successor.
- grok2 11y ago
  This explanation feels like the explanation by someone else on this page for why Skype failed on mobile...good insight.
- dman 11y ago
  Would you agree with this generalization? - Best companies in the world keep redefining the central problem they are attacking.
  - tn13 11y ago
    Yes and that redefinition very often comes from a drastic change in top management. Ballmar was different from Gates and Satya is different from Ballmar.
shadowmint 11y ago
Why do people keep reading TC? Here let me make an article... wait wait... ah... "Big Companies FAIL" that sounds like nice click bait. Now... hm, let's invent some stupid word to pad it out how about the 'Stack Fallacy'. Programmers will dig the 'stack' part. Yeah. Ship it! Seriously, this article is content free. People make products. Sometimes they work... sometimes they fail. If you pretend you have some magical insight into why they fail or succesd with gems of wisdom like: found it very difficult to succeed in what looks like a “trivial to build” app — social networks. and: The stack fallacy provides insights into why companies keep failing at the obvious things — things so close to their reach that they can surely build. The answer may be that the what is 100 times more important than the how. Then... wow. I don't even know what to say. Really? What you build is important? No kidding. Why is the top of the list this morning?
- puredemo 11y ago
  The technology world has grown so much in the last few years that any niche technology articles simply don't generate a widespread enough appeal, so what we see on YC today is highly generalized -- frequently to the point of sheer stupidity -- bullshit. >Why is the top of the list this morning? If anything, HN is basically more like "vaguely technical startup gossip" nowadays. Pretty useless, compared to something like /r/programming.
ljw1001 11y ago
Some insights perhaps, but the claim that this is "Why big companies keep failing" is way overblown
- anshublog 11y ago
  agreed. Its just hard to have title that says "Why some companies fail repeatedly". All of us are competing with "17 people under 30"... listicles.
ap22213 11y ago
An alternative (or maybe complimentary) theory is in Clayton M. Christensen's innovator's dilemma. Big companies build enormous revenue bases on certain types of technologies. Then they struggle to innovate because, by transitioning, they eat away at their existing revenue streams.
- HillRat 11y ago
  Actually, I don't remember that particular insight in TID; what you describe is what I personally call "the Blockbuster effect" -- the idea that a company can quite rationally refuse to adopt a disruptive innovation because its current financial structure is adapted to a particular size market with particular revenues, costs and liabilities. In Blockbuster's case, the risk was that aggressively taking on DVD-by-mail and later streaming would have eaten into its revenue structure (late fees); render operationally worthless its physical stores and thus sink their capital investment and liabilities underwater; and shift the market to a lower-revenue level that would not generate the cash flow necessary for Blockbuster's continued existence. Of course, they did go out of business while flailing about wildly, but to my mind that simply underscores the importance of knowing how to gracefully exit a market. Ironically, perpetually-beleaguered Yahoo isn't a bad model for that.
dpflan 11y ago
This article is funny - twisting together the ideas that a company specializes in a product in a specific market and that other companies can use the products / tools of other companies to develop their own unique products for a specific market and market development and competition. The "up-the-stack" company building something using products from "down-the-stack" has already entered a market, gained market share, and specialized in a market in which the "down-the-stack" company has no presence. Now, the "down-the-stack" company sees an example of a successful product that uses their technology that they know so well, but their company is not specialized for this product, so it hubristically does low-hanging-fruit-snatching to try to enter the new market. "Big companies keep failing" because they are not being innovative based upon what's mentioned in this article; they see an easy out and enter a market that already has an incumbent(s).
dkarapetyan 11y ago
Not so much stack as legacy. Have you seen legacy architectural decisions? They're impossible to get rid of. It's surprising how much the initial architecture can hinder change.
deleted 11y ago
[deleted]
tuke 11y ago
True enough, but there are also companies that were designed from the beginning to have vertical integration and control much of their business from beans to buildings. And of course I am thinking of Starbucks. (People don't really understand the technological story of Starbucks, which has a lot to do with their introduction of the vacuum packs to get their coffee across the country, and overcoming the challenge of brewing coffee on passenger jets.) Mostly for the better, they decided long ago to own as much of the stack as they could.
hyperpape 11y ago
I don't want to be too dismissive because something about the article rang true to me, but I don't know that I buy the whole central conceit that the idea of a stack can apply as universally as this article needs it to. Apple's networked services have often struggled. But are they really higher level than the things Apple succeeds at? Asking whether enormous distributed data stores are higher level than Mail.app just seems confused. It's different, and it brings new challenges, but are they part of the same stack? And is the data ingestion and sanitizing that Maps struggled with higher or lower level than the client that was basically ok? You can multiply these questions and I'm not sure you can get good answers.
- Throwaway23412 11y ago
  I would argue that networking services, backend services, and apps are indeed much higher level than the things Apple succeeds at (hardware, industrial design, and operating system). In fact, Steve Jobs never even wanted third-party apps on the iPhone (http://www.theguardian.com/technology/appsblog/2011/oct/24/steve-jobs-apps-iphone http://www.theguardian.com/technology/appsblog/2011/oct/24/s...). So, it's understandable why Apple has struggled with their networking services, backend services, and apps: such concerns weren't even remotely a part of Apple's DNA.
anshublog 11y ago
I am the author. Happy to answer any questions about my post.
- myth_buster 11y ago
  I think this could be turned into a book collating detailed case study in each of those cases where they failed with perhaps cases where companies have made it work like Google Fiber, Amazon's PAAS etc.
cbsmith 11y ago
This seems more than a bit flawed. It presumes companies or anyone else think these launches in to new markets are low risk. They are generally seen as anything but. There is hope that leveraging existing strengths will improve the odds, but only the idiots think they are certain to win.
jackgavigan 11y ago
> Product management is the art of knowing what to build. And in what order.
oautholaf 11y ago
A lot of the examples and counter-examples in the threads here are great, but Microsoft in the Windows era is a great counter-example here: from operating system to Office dominance. How did they crush Lotus and WordPerfect again?
vonklaus 11y ago
This article seems to have many correct pieces, but I don't think they coslesce to prove the point, or at least, not entirely. I don't think that manufacturing semiconductors are comparable to building maps. Apple should have done a better job with maps, and even though they do complex manufacturing, likely should have done worse at chip manufacturing. Iirc they brought in 3rd parties to help with the chip fab, and certainly spent more money building that core competency than maps. I believe the author is correct that the issues is companies not fully understanding, and consequently underestimate, what it takes to be successful in a different arena putside their cc. Google sees people as articles in a db. They dont understand people at all, they dont understand design as it relates to people, and they didn't understand that nobody needed another social network. They probably underinvested (initially) in G+ and it was not a great product. It didnt achieve critical mass quickly, and thus had no chance of growing as a docial platform ever. However, google is a lot more capable of creating something like this because they have all the core conpetencies down. I guess my takeaway is that the companies can in fact take these arenas, but they underestimate the challenge. So to use a drug dealing analogy, they try to start moving bricks amd kilos, instead of working their way up learning the market pushing dimes and quarters. They start too big, and when you fail big, you dont get the recovery of a smaller failure which affords small relaunches and features. Tldr big companies try to enter at the top, cant recover from huge public failures, and either exit or buy in
- anshublog 11y ago
  "I guess my takeaway is that the companies can in fact take these arenas, but they underestimate the challenge." Yes. And their core mistake is not understanding the real needs of the customer because they make assumptions in a familiar territory.
tokipin 11y ago
Nice observation. Another way to put it is that induction is harder than deduction. A related factor is that larger companies tend to be more specialized (formalized processes, specialists, focused teams/departments, and so on), meaning they can be prematurely optimized with respect to new goals and poorly equipped to conduct the necessary roaming.
cturner 11y ago
It's particularly funny when the stack fallacy is held by database providers, because databases are the wrong tool for most of the jobs they get used for at the moment. Current usage of the database uses it as a loose, adhoc, difficult-to-maintain, polling-based API between multiple applications. The future perspective looks back on our time, shaking its head at the way people use databases for everything in the same way that we shake our heads at bloodletting. Oracle's business model is (1) convincing people to use platforms they shouldn't be using and then (2) selling the victims ongoing hacks and services to work around the limitations of the model. Amazon's software services won't be build on a database. They'll be built using a decentralised messaging platform.
a-robinson 11y ago
The author claims that "Stack fallacy is the mistaken belief that it is trivial to build the layer above yours", but then says that IBM was wrong when they "happily allowed Microsoft to own the OS market". Wasn't IBM a classic case of not trying to build the layer above them on the stack? The Wikipedia page on IBM PC DOS even claims that their "radical break from company tradition of in-house development was one of the key decisions that made the IBM PC an industry standard".
- Sammi 11y ago
  Oh IBM tried building it: http://arstechnica.com/business/2013/11/half-an-operating-system-the-triumph-and-tragedy-of-os2/2/ http://arstechnica.com/business/2013/11/half-an-operating-sy...
  - Qwertious 11y ago
    OS/2 was too little too late - by the time they realised what was going on, Microsoft was already a new major player. Didn't stop Microsoft from going out of their way to sabotage them, though.
    - yuhong 11y ago
      In fact, the history of 32-bit OS/2 is more complex than that, and involved MS a lot more than this. It used to be my favorite topic.
fforflo 11y ago
As a comment on TC says: "What the article is referring to as stack fallacy is the work of Physics Nobel Laureate Philip Anderson: https://web2.ph.utexas.edu/~wktse/Welcome_files/More_Is_Different_Phil_Anderson.pdf https://web2.ph.utexas.edu/~wktse/Welcome_files/More_Is_Diff... Let's give credit where it due please."
shim2k 11y ago
Peter Lynch writes about it in his book "One Up on Wall Street". He calls it 'Diworsification'.
annnnd 11y ago
> The bottleneck for success often is not knowledge of the tools, but lack of understanding of the customer needs. THIS! +1000! I would even leave out "often", or at least replace it with "usually".
peter303 11y ago
The counter example would be Apple when they introduced music players and music distribution. That was far away from their core business of PCs. The smartphone was less of a stretch, because it was a potential extension of the iTouch comunications computer.
LCDninja 11y ago
When reading the article I was reminded about Warren Buffets message re identifying and staying within your "circle of competence."
a-dub 11y ago
That's why when I build a company, I'm going to use a hash table.

408 points by walterclifford 11y ago

ogezi 11y ago

A company should always focus on your strengths if not it'll be both overstretched and unsuccessful. Great read.

justicezyx 11y ago
I do not think that's the correct conclusion. The conclusion should be that when venture into new areas, be sure to understand the market first.
- 2np 11y ago
  Or, just because you understand the underlying technology doesn't mean you understand the market, and your previous experience will get in the way of actually understanding the market. If you want to go down the tech stack, though, you might be successful because you're part of the market for that tech.
  - anshublog 11y ago
    Exactly. When you go down the stack like Amazon did with AWS, you know your customer (often you). You also know the key pain points and gaps in the market. This is especially so in "as a service" markets.
  - anshublog 11y ago
    Exactly. When you go down the stack like Amazon did with AWS, you know your customer (often you). You also know the key pain points and gaps in the market. This is especially so in "as a service" markets.

adevine 11y ago

I don't buy the arguments in this article. For example, the whole part about why Google failed with Google+ is just wrong IMO. It wasn't that Google wasn't capable of building a good social network. If anything, I (and most people I know) preferred the design and interface of Google+. The problem was that Facebook already had a huge head start, and all your friends were already there. Facebook was "good enough", and there wasn't a big enough incentive to want to switch to Google+. If anything, large companies often miss out on new trends and changes in business and technology, but it's not solely because building that one new layer "up the stack" is so technically hard or different.

riyadparvez 11y ago
Also there is this thing called 'network effect' which plays a huge role for social networks. EDIT: Why the downvote?
- danieltillett 11y ago
  I didn't down vote you, but my guess it is the tone of your comment. You are of course 100% right that for a general social network there can only be one. Until Facebook messes up it won't be possible to build a new general social network - maybe News Corp will buy Facebook and an opening will arise.
rajangdavis 11y ago
I think this example adds truth to the author's argument in that there have been a few social media apps like Instagram and Snapchat that took off in ways the Google Plus never did. I do agree that there was little impetus to move to Google Plus from Facebook, but there could have been features that would have made it enough of a differentiator that people could have moved over to Google Plus. It's not like Google lacked the ability to scale out or provide such a service.
- adevine 11y ago
  I would argue that Snapchat and Instagram occupy VERY different niches than Facebook. Most importantly, virtually everyone who uses Snapchat or Instagram also uses FB. Google thought they had enough clout to win the "general all purpose category" social network - they were trying to compete directly against FB.

kazinator 11y ago

This stack fallacy sounds very familiar. Oh, we will just have a few system calls like open, close, read and write, some TTY and credentials related stuff, a bit of signal handling, process control with fork, exect and wait ... writing a shell language on top will practically just be a footnote.

anshublog 11y ago
Yes many open source projects suffer from this too. We as engineers (even former ones) often assume the what to build is "obvious". This is often not true, if ever. The best open source projects are ones where engineers are the end users (customers).
- marshray 11y ago
  Perhaps this is a feeling that, as experienced implementers, we have a big head start on the hard parts. Maybe this is another form of the "solution looking for a problem" phenomenon.

anjc 11y ago

I'd like some solid examples of what companies confirmed perceptions of competitors were in the same vertical, versus the reality. Because even the author references competency-based views of competitive advantage, but for some reason ignores resource based views, and ignores the fact that companies might be aware of their competences. That is to say, I'm sure that large companies tend to mostly be aware of what their competences are based on the resources and knowledge that they have. If they don't have marketing departments that have analyzed the ERP market, sales teams with ERP training, tech departments with key HR, key knowledge etc etc, then I'm certain they are very well aware of this. Maybe some companies have had marketing missteps and have made poor strategic and competitive decisions, however, but I really doubt that it's due to a lack of introspection or simple analysis as described. Also, IBM didn't "think nothing much" of the software layer. They misunderstood the nature of power in the supply chain, and most importantly, didn't solidify their position within the supply chain while they were dominant.

anshublog 11y ago
They think they know their competencies (database systems in one case, virtualization in another) but often they misunderstand what customers really value up the stack - in this example, Workday customers really don't care if it runs Oracle or MySQL or non RDBMS at all. In AWS case, we don't care for kvm vs VMware wars. (I am the author of the TechCrunch post.)

gregdoesit 11y ago

From my experience the same (often random) reason that makes a company succeeds, then becomes their DNA, and finally can make them fail. I saw this happen with Skype where I worked a couple of years. The company succeeded because of P2P: we grew with little infrastructure to reach 200M+ people. P2P became our DNA, rooted deep within (almost) every core component. Then came the new wave of mobile messaging apps. We reacted... with a P2P messaging solution. It was obvious this wasn't working - you sent a message to someone from Skype for iPhone, and they got it... sometime. We knew to have a chance against Whatsapp and other messaging apps we needed server based messaging, so we built it. It took 3 years. Yes, it took this long to get rid of the P2P code from just the messaging components from the 20+ Skype products - we had 1,000+ engineers and 50+ internal teams by the end which significantly slowed things down. When we were done and popped the champagne - no one really cared. And yes, the source code is still full of P2P references and workarounds to this date.

im2w1l 11y ago
Why was it hard to deliver messages to iphone using p2p? I mean I guess they couldn't open ports and listen for messages, but that should have been true for plenty of NATed systems too right?
- joshyeager 11y ago
  We used Skype IM for a long time (inertia). It wasn't just the iPhone. Skype message delivery was erratic on all platforms. It got better, but even now there are occasional delivery delays when I chat with someone who hasn't switched over (we use Slack now).
  - gregdoesit 11y ago
    Yes, there are still traces of the p2p code. Most of messaging has been migrated to server based back ends, but there are still some hybrid solutions in place. Funny thing the company DNA, P2P is still haunting Skype.
    - 11y ago

mwnz 11y ago

Do big companies really keep failing? I'm failing to see the evidence of that assertion.

sawyer 11y ago
Failing to gain traction upstream, not failing to keep the lights on.

libertymcateer 11y ago

Apple is not vertically integrated - Wikipedia entries to the contrary notwithstanding. It is a grossly inaccurate statement. Up until very recently, Apple didn't own a single factory - how can one possibly claim that they are vertically integrated if they don't own their own means of production? Apple is a fantastically successful software and industrial design company. The vast majority of their production is outsourced. This is not vertical integration. Additionally, actually, Apple has tremendous amounts of hugely successful and popular software. Though I dig the underlying point of this article, that product management is hard, I think the examples are less than good.

vacri 11y ago
Where do you draw the line? If a company does own the factory that builds their widgets, do you claim they're not vertically integrated because they don't own the mines that get the materials, nor the transport network to get it to and from the factory? At the other end, do they need to own the bank with which they take payments from customers? You don't need to own all elements in a vertical stack to be vertically integrated.
- josu 11y ago
  Apple sells iphones. Who makes the iphones? Not Apple.
  - beambot 11y ago
    Apple designs iPhones. They manage the fabrication of parts very closely and manage their assembly and QA more closely than most of their competitors (like, big enough to make demands of Foxconn or cause them to build specialized factories). Then they sell the phones directly to consumers. They are (literally) on the Wikipedia page as an example of a vertically integrated company: https://en.wikipedia.org/wiki/Vertical_integration https://en.wikipedia.org/wiki/Vertical_integration I'm not saying you're wrong... but it doesn't look like you're right.
  - vacri 11y ago
    You're playing fast and loose with the definition of 'make' there. 'Physically assemble' is not the only element of 'make'.

libertymcateer 11y ago

marshray 11y ago

> “Can we compete with Intel or SAP?” Well for one thing we know that Intel spends several $billion to open a new semiconductor plant and has a dozen of them already. https://en.wikipedia.org/wiki/List_of_Intel_manufacturing_sites https://en.wikipedia.org/wiki/List_of_Intel_manufacturing_si... Whereas SAP is, well, a lot of software. Which is something, but Intel needs to make a lot of software too, and chip designs are in some ways a specialized form of software. So I think in some sense Intel is strictly more challenging to replicate than SAP. (But this is probably just my misunderestimation talking. :-)

at5 11y ago
You're actually quite right. Capital intensiveness is a barrier to entry. That's why software businesses rise rapidly and fall rapidly at a greater frequency than other types of businesses.

anonymousguy 11y ago

The solution to stack fallacy is simple but really counter-intuitive. All of the mentioned examples, I mean every single one, indicate a business trying to force its way into the higher level through business channels. For example, when a business wants into a higher level they make it a business priority to create a new product and attempt to drive the priorities of this next level product through their business objectives. That is an epic fail. It is important to instead concede that you don't know the needs of the consumers in the higher level, and if you think you do it is because you are guessing. The only way avoid the problem is to not attempt to move into the higher level, at least not intentionally and not through business priorities. This is extremely counter-intuitive because there are generally fewer expenses and greater market frequency at each higher level, which means superior revenue potential. Businesses exist to make money and to ignore moving up to the higher level means denying this potential (vast) revenue source. This doesn't mean you can't move into the higher level of the stack and be really good at it. It just means you cannot do so both intentionally and as a business objective. The solution is to double-down on where you already are with what you are already good at and focus on product quality of your existing products. Continue to improve where you are already good. Improvements and enhancements to existing products can gradually yield the next level as the improvements progressively open new potential in an evolutionary fashion. While getting to the next level this way is much slower it is also risk reduced and continues to associate your brand with quality and consume satisfaction. This will only work, though, if the goal is improving the current product and not acquiring revenue in that desired higher level. Think evolution and not revolution. It has to be a gradual, almost accidental, increase of capability based on meeting current consumer needs.

PhilWright 11y ago
I don't think this works in practice. I cannot see Oracle slowly adding extra features to its database until it becomes an actual CRM system. A CRM is not just a solution to a different business problem but also a logically different layer from a development perspective. You would have to build it as a product that sits on top of the database product. But as a CRM system it has no value to the end customer unless it has a certain minimum level of functionality. So although you can add features gradually and evolve your CRM offering you will not have a single customer until you have actually added enough features that make it actually usable. Nobody will buy your gradually evolving system until several years have passed and it works for some actual customers. In which case you have already spend alot of money and effort.

tn13 11y ago

There was a brilliant essay by an Indian politician few years back after his party lost the elections. Later in lecture he explain why political parties and large companies have so much in common when it comes to failing. His basic logic was that - Success depends on processes - Processes even though might be thought of as abstract in reality are function of people at top. - Company gets successful because some bright guy is the rebel, he questions status quo, persists and succeeds. - As time goes by, the rebellious ideas actually become conservative ideas. The rebel is now on top. As his ideas fade he struggles to stay on top. - He recruits people who see the world through him, he builds processes that enforce that vision. - This makes it difficult for the truth to be visible to the top management. - By the time failure is visible it is hard to turn around the ship. - IN SHORT: Companies/Nations fail because someone at top did not know when to quit. - In the end that rebel turned conservative becomes bitter. He thinks the world owed him something for what he achieved. He explained who USSR examples. How a genetic scientist got promoted because his fake research re-enforced something that Stalin had said long back and his peers were scared to point out the fact because it might get perceived as anti-Stalin. I observed Blackberry very closely and it resonated to me so much. The founders at one point blamed people for using iphone and not blackberry. Best companies in the world are seem to be those where their top leaders quit at their peak to make way for their successor.

grok2 11y ago
This explanation feels like the explanation by someone else on this page for why Skype failed on mobile...good insight.
dman 11y ago
Would you agree with this generalization? - Best companies in the world keep redefining the central problem they are attacking.
- tn13 11y ago
  Yes and that redefinition very often comes from a drastic change in top management. Ballmar was different from Gates and Satya is different from Ballmar.

shadowmint 11y ago

Why do people keep reading TC? Here let me make an article... wait wait... ah... "Big Companies FAIL" that sounds like nice click bait. Now... hm, let's invent some stupid word to pad it out how about the 'Stack Fallacy'. Programmers will dig the 'stack' part. Yeah. Ship it! Seriously, this article is content free. People make products. Sometimes they work... sometimes they fail. If you pretend you have some magical insight into why they fail or succesd with gems of wisdom like: found it very difficult to succeed in what looks like a “trivial to build” app — social networks. and: The stack fallacy provides insights into why companies keep failing at the obvious things — things so close to their reach that they can surely build. The answer may be that the what is 100 times more important than the how. Then... wow. I don't even know what to say. Really? What you build is important? No kidding. Why is the top of the list this morning?

puredemo 11y ago
The technology world has grown so much in the last few years that any niche technology articles simply don't generate a widespread enough appeal, so what we see on YC today is highly generalized -- frequently to the point of sheer stupidity -- bullshit. >Why is the top of the list this morning? If anything, HN is basically more like "vaguely technical startup gossip" nowadays. Pretty useless, compared to something like /r/programming.

ljw1001 11y ago

Some insights perhaps, but the claim that this is "Why big companies keep failing" is way overblown

anshublog 11y ago
agreed. Its just hard to have title that says "Why some companies fail repeatedly". All of us are competing with "17 people under 30"... listicles.

ap22213 11y ago

An alternative (or maybe complimentary) theory is in Clayton M. Christensen's innovator's dilemma. Big companies build enormous revenue bases on certain types of technologies. Then they struggle to innovate because, by transitioning, they eat away at their existing revenue streams.

HillRat 11y ago
Actually, I don't remember that particular insight in TID; what you describe is what I personally call "the Blockbuster effect" -- the idea that a company can quite rationally refuse to adopt a disruptive innovation because its current financial structure is adapted to a particular size market with particular revenues, costs and liabilities. In Blockbuster's case, the risk was that aggressively taking on DVD-by-mail and later streaming would have eaten into its revenue structure (late fees); render operationally worthless its physical stores and thus sink their capital investment and liabilities underwater; and shift the market to a lower-revenue level that would not generate the cash flow necessary for Blockbuster's continued existence. Of course, they did go out of business while flailing about wildly, but to my mind that simply underscores the importance of knowing how to gracefully exit a market. Ironically, perpetually-beleaguered Yahoo isn't a bad model for that.

dpflan 11y ago

This article is funny - twisting together the ideas that a company specializes in a product in a specific market and that other companies can use the products / tools of other companies to develop their own unique products for a specific market and market development and competition. The "up-the-stack" company building something using products from "down-the-stack" has already entered a market, gained market share, and specialized in a market in which the "down-the-stack" company has no presence. Now, the "down-the-stack" company sees an example of a successful product that uses their technology that they know so well, but their company is not specialized for this product, so it hubristically does low-hanging-fruit-snatching to try to enter the new market. "Big companies keep failing" because they are not being innovative based upon what's mentioned in this article; they see an easy out and enter a market that already has an incumbent(s).

dkarapetyan 11y ago

Not so much stack as legacy. Have you seen legacy architectural decisions? They're impossible to get rid of. It's surprising how much the initial architecture can hinder change.

deleted 11y ago

[deleted]

tuke 11y ago

True enough, but there are also companies that were designed from the beginning to have vertical integration and control much of their business from beans to buildings. And of course I am thinking of Starbucks. (People don't really understand the technological story of Starbucks, which has a lot to do with their introduction of the vacuum packs to get their coffee across the country, and overcoming the challenge of brewing coffee on passenger jets.) Mostly for the better, they decided long ago to own as much of the stack as they could.

hyperpape 11y ago

I don't want to be too dismissive because something about the article rang true to me, but I don't know that I buy the whole central conceit that the idea of a stack can apply as universally as this article needs it to. Apple's networked services have often struggled. But are they really higher level than the things Apple succeeds at? Asking whether enormous distributed data stores are higher level than Mail.app just seems confused. It's different, and it brings new challenges, but are they part of the same stack? And is the data ingestion and sanitizing that Maps struggled with higher or lower level than the client that was basically ok? You can multiply these questions and I'm not sure you can get good answers.

Throwaway23412 11y ago
I would argue that networking services, backend services, and apps are indeed much higher level than the things Apple succeeds at (hardware, industrial design, and operating system). In fact, Steve Jobs never even wanted third-party apps on the iPhone (http://www.theguardian.com/technology/appsblog/2011/oct/24/steve-jobs-apps-iphone http://www.theguardian.com/technology/appsblog/2011/oct/24/s...). So, it's understandable why Apple has struggled with their networking services, backend services, and apps: such concerns weren't even remotely a part of Apple's DNA.

anshublog 11y ago

I am the author. Happy to answer any questions about my post.

myth_buster 11y ago
I think this could be turned into a book collating detailed case study in each of those cases where they failed with perhaps cases where companies have made it work like Google Fiber, Amazon's PAAS etc.

cbsmith 11y ago

This seems more than a bit flawed. It presumes companies or anyone else think these launches in to new markets are low risk. They are generally seen as anything but. There is hope that leveraging existing strengths will improve the odds, but only the idiots think they are certain to win.

jackgavigan 11y ago

> Product management is the art of knowing what to build. And in what order.

oautholaf 11y ago

A lot of the examples and counter-examples in the threads here are great, but Microsoft in the Windows era is a great counter-example here: from operating system to Office dominance. How did they crush Lotus and WordPerfect again?

vonklaus 11y ago

This article seems to have many correct pieces, but I don't think they coslesce to prove the point, or at least, not entirely. I don't think that manufacturing semiconductors are comparable to building maps. Apple should have done a better job with maps, and even though they do complex manufacturing, likely should have done worse at chip manufacturing. Iirc they brought in 3rd parties to help with the chip fab, and certainly spent more money building that core competency than maps. I believe the author is correct that the issues is companies not fully understanding, and consequently underestimate, what it takes to be successful in a different arena putside their cc. Google sees people as articles in a db. They dont understand people at all, they dont understand design as it relates to people, and they didn't understand that nobody needed another social network. They probably underinvested (initially) in G+ and it was not a great product. It didnt achieve critical mass quickly, and thus had no chance of growing as a docial platform ever. However, google is a lot more capable of creating something like this because they have all the core conpetencies down. I guess my takeaway is that the companies can in fact take these arenas, but they underestimate the challenge. So to use a drug dealing analogy, they try to start moving bricks amd kilos, instead of working their way up learning the market pushing dimes and quarters. They start too big, and when you fail big, you dont get the recovery of a smaller failure which affords small relaunches and features. Tldr big companies try to enter at the top, cant recover from huge public failures, and either exit or buy in

anshublog 11y ago
"I guess my takeaway is that the companies can in fact take these arenas, but they underestimate the challenge." Yes. And their core mistake is not understanding the real needs of the customer because they make assumptions in a familiar territory.

tokipin 11y ago

Nice observation. Another way to put it is that induction is harder than deduction. A related factor is that larger companies tend to be more specialized (formalized processes, specialists, focused teams/departments, and so on), meaning they can be prematurely optimized with respect to new goals and poorly equipped to conduct the necessary roaming.

cturner 11y ago

It's particularly funny when the stack fallacy is held by database providers, because databases are the wrong tool for most of the jobs they get used for at the moment. Current usage of the database uses it as a loose, adhoc, difficult-to-maintain, polling-based API between multiple applications. The future perspective looks back on our time, shaking its head at the way people use databases for everything in the same way that we shake our heads at bloodletting. Oracle's business model is (1) convincing people to use platforms they shouldn't be using and then (2) selling the victims ongoing hacks and services to work around the limitations of the model. Amazon's software services won't be build on a database. They'll be built using a decentralised messaging platform.

a-robinson 11y ago

The author claims that "Stack fallacy is the mistaken belief that it is trivial to build the layer above yours", but then says that IBM was wrong when they "happily allowed Microsoft to own the OS market". Wasn't IBM a classic case of not trying to build the layer above them on the stack? The Wikipedia page on IBM PC DOS even claims that their "radical break from company tradition of in-house development was one of the key decisions that made the IBM PC an industry standard".

Sammi 11y ago
Oh IBM tried building it: http://arstechnica.com/business/2013/11/half-an-operating-system-the-triumph-and-tragedy-of-os2/2/ http://arstechnica.com/business/2013/11/half-an-operating-sy...
- Qwertious 11y ago
  OS/2 was too little too late - by the time they realised what was going on, Microsoft was already a new major player. Didn't stop Microsoft from going out of their way to sabotage them, though.
  - yuhong 11y ago
    In fact, the history of 32-bit OS/2 is more complex than that, and involved MS a lot more than this. It used to be my favorite topic.

fforflo 11y ago

As a comment on TC says: "What the article is referring to as stack fallacy is the work of Physics Nobel Laureate Philip Anderson: https://web2.ph.utexas.edu/~wktse/Welcome_files/More_Is_Different_Phil_Anderson.pdf https://web2.ph.utexas.edu/~wktse/Welcome_files/More_Is_Diff... Let's give credit where it due please."

shim2k 11y ago

Peter Lynch writes about it in his book "One Up on Wall Street". He calls it 'Diworsification'.

annnnd 11y ago

> The bottleneck for success often is not knowledge of the tools, but lack of understanding of the customer needs. THIS! +1000! I would even leave out "often", or at least replace it with "usually".

peter303 11y ago

The counter example would be Apple when they introduced music players and music distribution. That was far away from their core business of PCs. The smartphone was less of a stretch, because it was a potential extension of the iTouch comunications computer.

LCDninja 11y ago

When reading the article I was reminded about Warren Buffets message re identifying and staying within your "circle of competence."

a-dub 11y ago

That's why when I build a company, I'm going to use a hash table.