5 ms·

I've found that slowdown from tech debt killed as many companies as any other issue. It's usually caused by business owners constantly pivoting, but being too s

by izacus 4y ago

I've found that slowdown from tech debt killed as many companies as any other issue. It's usually caused by business owners constantly pivoting, but being too slow on the pivot and too slow to bring customer wishes to fruition (due to poor technical decisions and tech debt) is probably one of the top 5 reasons for dead companies I've seen.

brainzap 4y ago
we need data on this
jacquesm 4y ago
That's a good point, tech debt can be a killer. But the more common pattern that I've seen is that companies that accumulate tech debt but that are doing well commercially eventually gather enough funds to address the tech debt, the companies that try to get the tech 'perfect' the first time out of the gate lose a lot more time, and so run a much larger chance of dying. The optimum is to allow for some tech debt to accumulate but to address it periodically, either by simply abandoning those parts by doing small, local rewrite (never the whole thing all at once, that is bound to fail spectacularly) or by having time marked out for refactoring. The teams that drown in tech debt tend to have roadmaps that are strictly customer facing work, that can get you very far but in the end you'll stagnate in ways that are not easy to fix, technical work related to doing things right once you know exactly what you need pays off.
- p_l 4y ago
  That's missing the case where the tech debt results in lowered commercial performance, as things necessary to keep customers happy enough to provide the cash flow are getting harder and harder.
  - jghn 4y ago
    They’re saying that in their experience this isn’t nearly as common as it is typically portrayed.
    - nickcox 4y ago
      > They’re saying that in their experience this isn’t nearly as common as it is typically portrayed. How would you know if the poor commercial performance was due to tech debt or not though? It's the intangibility of tech debt that makes it so insidious.
      - jghn 4y ago
        Can't speak for the person I was citing. But I think I get what they're saying. My personal experience has been that tech debt is more often caused by business level decisions and not engineering decisions. Deadline on this contract is next week so let's ship what we got and worry about it later. Hey, good news everyone we just pivoted 180 degrees so let's try to salvage what we've got. So yes it very well might be that a mountain of tech debt was the final nail in the coffin. But why was that tech debt there in the first place? I was understanding the GP as saying they saw business decisions leading to poor engineering instead of engineering just doing dumb things on their own. I've seen plenty of examples of the latter but a lot more of the former in my travels.
- jakey_bakey 4y ago
  Exactly this. People forget that the point of the metaphor is that debt is a tool you use to grow faster. Credit card debt (e.g. sloppy code and test-free critical backend processes) is pretty bad and should be paid down ASAP. Mortgage debt (e.g. no UI tests on the front-end) is quite safe and you can kick the can down the road.
  - azemetre 4y ago
    In my experience when you don't design and deliver code with testing or accessibility in mind, you end rewriting entire components. This all drastically adds to the end costs. Most leadership thinks this is "efficient" but it's not really. If you do it correct the first time you can consistently deliver features throughout the entire year rather than having to take several months to quickly duct tape everything from falling apart. I never liked the "debt" metaphor. If a housing developer neglected to build a proper foundation, would you call that "debt?" I feel like it's very similar, it's bad metaphor for a concept that has very little to do with finance.
- fullstackchris 4y ago
  The double challenge here is doing this all whilst essentially keeping it in the background out of any customer sight. Even if you know exactly what you need after a while from a business perspective, you still need to reimplement it in a way that doesn't cause your product / service / platform to lose customers. I find this always to be an extreme challenge. It's a bit of a treadmill too: doing it this way (without causing breaking changes) certainly takes longer too. So it all piles up into a big messy stack of work :)
- bcrosby95 4y ago
  Maybe once you get to that stage it doesn't really matter. Maybe if you're going for a billion dollar earth shaking idea, it doesn't really matter. However, I've worked for a small company for quite a while now. We've had several successful projects and several failures. In my experience, technical debt taken too early can easily double the time it takes you to find out if a project is a dud. That matters to us. My general rule is: push off technical debt as late as you can. Aways leave code slightly better than you found it. Fix problems as you recognize them. I think a big mistake developers make is thinking "make code better" should be on some roadmap. You should be making code better every time you touch it. Nothing about writing a new feature says that you have to integrate it in the sloppiest way possible.
  - bombcar 4y ago
    Even just #commenting on weirdness you discovered while working on tech-debt code can be invaluable; a little note on the two hours you spent on it could save days later when trying to figure out why something isn't working. Sometimes the problems can't be fixed right then, but you can mark them so later when it does break you have a hint as to what is going wrong.
  - JamesBarney 4y ago
    What are the technical debt issues you've run into that've crippled your development velocity?
    - _vvhw 4y ago
      Technical debt has the second order effect of crippling morale.
  - jimbokun 4y ago
    > I think a big mistake developers make is thinking "make code better" should be on some roadmap. You should be making code better every time you touch it. Nothing about writing a new feature says that you have to integrate it in the sloppiest way possible. I vehemently agree. One of my first jobs was working for a mathematician at a bank, who could code well enough to compute his mathematical ideas, but not a software engineer so hired me to do more of the coding for his team. He would say "Jim, just get this done but don't spend time making it fancy." In other words, don't spend time refactoring or cleaning up the code, expressed in his own words. I would say "sure" and then proceed to refactor and clean up the code as I went. It took less time than it would have to write the code "ugly" then deal with the accumulated tech debt, and I finished everything on time so he was happy.
- theptip 4y ago
  It’s one of those systemic health type things. It’s really hard to die of tech debt on its own, but if you move slower, you’ll die more often from other shocks. Another way of thinking about it is that you have N months of runway, and based on your velocity you can pull off a pivot in M months, and the more tech debt, the more time it will take to successfully pivot. If you don’t have a full pivot worth of runway remaining, and you need to pivot, you die. (Of course this oversimplifies by holding pivot magnitude equal but hopefully this illustrates the point.) I do agree that away from the margin, companies that are incredibly successful can afford to punt harder on tech debt. I suppose “know thyself” might be useful advice here; it’s probably not good advice for the median startup to ignore tech debt completely IMO. I think the main point though is to optimize for agility; tech debt can let you move faster in the short and even medium term, so sometimes it’s right to tactically take on some debt. But not so much that you get bogged down later; make sure you carve out time to fix the stuff that starts to be painful.
- nonameiguess 4y ago
  I feel like the biggest problem from a business strategy perspective is all we have are these personal opinions and gut feels. Even this article mentions having done 20 code audits, but presents nothing but qualitative findings. Ideally, some business school out there would be embedding researchers in randomly selected startups to know for sure how often you fail because of tech debt versus failing because of worrying too much about tech debt. That's an empirical question, yet all we get are informed expert opinions, but no auditable, reproducible research evidence. It's all so unscientific. Not to say you're wrong, but we have no real way of even deciding. All I can do is lean on my own experience, but I've seen nowhere near every product team out there and the ones I have seen are nothing close to randomly sampled or blinded.
spdionis 4y ago
Well, why are those companies pivoting so often in the first place? Isn't the root cause probably in GP's list?
phphphphp 4y ago
I’m not sure I agree. Technical debt is a symptom, it’s the consequence of bad management that leads to working on the wrong things. If you’re running a startup and haven’t yet found your feet in terms of a product offering, and you’re building your product(s) in such a way that technical debt builds up through continuously layering half-baked on half-baked, it’s indicative that you’re not actually pivoting and not actually evolving, you’re just adding new half-baked ideas to a half-baked system… and being able to do that at twice the speed isn’t going to address the real problem: half-baked ideas don’t make a product, whether that’s 10 half-baked ideas or 100. My experience is that any company in which evolution/experiments/pivoting is constrained within the boundaries of what already exists because of the sunk cost fallacy has made a grave error at a leadership level, not at a code level. If you can’t validate something without mashing more code into your system, that’s the problem to address. I’ve seen companies with horrendous tech debt die, and you could certainly frame their death as being a consequence of the tech debt (“if they had just got the perfect system…”) but that assumes the perfect system would somehow prevent them from making the mistakes that got them there in the first place. It wouldn’t. The technical debt is an expression of their mistakes, not the cause. You could dump the perfect system at their feet and they’d be surrounded by garbage again a few years from now.
- icedchai 4y ago
  I worked at a company that was mired in tech debt. At least 4 different UI frameworks were in use, one of which was totally not supported. Multiple versions of the app were left accessible, with links from the new to the old, because the new version was not feature complete. "Feature flags" were expressed in at least 3 different ways. It was a nightmare to figure out if something was on or off, and why. The back end was based on a non supported language version, with several old, deprecated third-party packages as a result. The company appeared organized, superficially, but at the lower levels of implementation it was a total dumpster fire. They were constantly "pivoting", but leaving the old junk around.
  - redisman 4y ago
    There’s tech debt and then there’s poor engineering leadership. There’s no valid reason for a startup looking for market fit to switch frameworks or feature flags multiple times unless you’re just being clueless and looking for silver bullets. Just pick a few “boring” technologies and you’ll be perfectly capable of building anything “web” for at least a decade without messing around .
new_stranger 4y ago
And keeping 100% of all features instead of removing the least-used features as you add new ones to keep tech debt from growing indefinitely and reaching a point where new features take months to ship.
JamesBarney 4y ago
What killed them was they never found PMF. Eventually the tech debt slowed them down so they couldn't take as many swings at finding PMF. But in the counterfactual if they'd tried really hard to avoid tech debt that would have slowed them down at the beginning, not to mention there are plenty of organizations that will write very complex abstract code to avoid tech debt, and end up making the code base incredibly painful to work with. So overall did get they get less swings? I've worked on a lot of old code bases and the biggest issues I've run into, issues that crippled development velocity, were 95% boneheaded decisions and overengineering. And never the types of code quality issues someone like Uncle Bob talks about in Clean Code.

by izacus 4y ago

brainzap 4y ago

we need data on this

jacquesm 4y ago

That's a good point, tech debt can be a killer. But the more common pattern that I've seen is that companies that accumulate tech debt but that are doing well commercially eventually gather enough funds to address the tech debt, the companies that try to get the tech 'perfect' the first time out of the gate lose a lot more time, and so run a much larger chance of dying. The optimum is to allow for some tech debt to accumulate but to address it periodically, either by simply abandoning those parts by doing small, local rewrite (never the whole thing all at once, that is bound to fail spectacularly) or by having time marked out for refactoring. The teams that drown in tech debt tend to have roadmaps that are strictly customer facing work, that can get you very far but in the end you'll stagnate in ways that are not easy to fix, technical work related to doing things right once you know exactly what you need pays off.

p_l 4y ago
That's missing the case where the tech debt results in lowered commercial performance, as things necessary to keep customers happy enough to provide the cash flow are getting harder and harder.
- jghn 4y ago
  They’re saying that in their experience this isn’t nearly as common as it is typically portrayed.
  - nickcox 4y ago
    > They’re saying that in their experience this isn’t nearly as common as it is typically portrayed. How would you know if the poor commercial performance was due to tech debt or not though? It's the intangibility of tech debt that makes it so insidious.
    - jghn 4y ago
      Can't speak for the person I was citing. But I think I get what they're saying. My personal experience has been that tech debt is more often caused by business level decisions and not engineering decisions. Deadline on this contract is next week so let's ship what we got and worry about it later. Hey, good news everyone we just pivoted 180 degrees so let's try to salvage what we've got. So yes it very well might be that a mountain of tech debt was the final nail in the coffin. But why was that tech debt there in the first place? I was understanding the GP as saying they saw business decisions leading to poor engineering instead of engineering just doing dumb things on their own. I've seen plenty of examples of the latter but a lot more of the former in my travels.
jakey_bakey 4y ago
Exactly this. People forget that the point of the metaphor is that debt is a tool you use to grow faster. Credit card debt (e.g. sloppy code and test-free critical backend processes) is pretty bad and should be paid down ASAP. Mortgage debt (e.g. no UI tests on the front-end) is quite safe and you can kick the can down the road.
- azemetre 4y ago
  In my experience when you don't design and deliver code with testing or accessibility in mind, you end rewriting entire components. This all drastically adds to the end costs. Most leadership thinks this is "efficient" but it's not really. If you do it correct the first time you can consistently deliver features throughout the entire year rather than having to take several months to quickly duct tape everything from falling apart. I never liked the "debt" metaphor. If a housing developer neglected to build a proper foundation, would you call that "debt?" I feel like it's very similar, it's bad metaphor for a concept that has very little to do with finance.
fullstackchris 4y ago
The double challenge here is doing this all whilst essentially keeping it in the background out of any customer sight. Even if you know exactly what you need after a while from a business perspective, you still need to reimplement it in a way that doesn't cause your product / service / platform to lose customers. I find this always to be an extreme challenge. It's a bit of a treadmill too: doing it this way (without causing breaking changes) certainly takes longer too. So it all piles up into a big messy stack of work :)
bcrosby95 4y ago
Maybe once you get to that stage it doesn't really matter. Maybe if you're going for a billion dollar earth shaking idea, it doesn't really matter. However, I've worked for a small company for quite a while now. We've had several successful projects and several failures. In my experience, technical debt taken too early can easily double the time it takes you to find out if a project is a dud. That matters to us. My general rule is: push off technical debt as late as you can. Aways leave code slightly better than you found it. Fix problems as you recognize them. I think a big mistake developers make is thinking "make code better" should be on some roadmap. You should be making code better every time you touch it. Nothing about writing a new feature says that you have to integrate it in the sloppiest way possible.
- bombcar 4y ago
  Even just #commenting on weirdness you discovered while working on tech-debt code can be invaluable; a little note on the two hours you spent on it could save days later when trying to figure out why something isn't working. Sometimes the problems can't be fixed right then, but you can mark them so later when it does break you have a hint as to what is going wrong.
- JamesBarney 4y ago
  What are the technical debt issues you've run into that've crippled your development velocity?
  - _vvhw 4y ago
    Technical debt has the second order effect of crippling morale.
- jimbokun 4y ago
  > I think a big mistake developers make is thinking "make code better" should be on some roadmap. You should be making code better every time you touch it. Nothing about writing a new feature says that you have to integrate it in the sloppiest way possible. I vehemently agree. One of my first jobs was working for a mathematician at a bank, who could code well enough to compute his mathematical ideas, but not a software engineer so hired me to do more of the coding for his team. He would say "Jim, just get this done but don't spend time making it fancy." In other words, don't spend time refactoring or cleaning up the code, expressed in his own words. I would say "sure" and then proceed to refactor and clean up the code as I went. It took less time than it would have to write the code "ugly" then deal with the accumulated tech debt, and I finished everything on time so he was happy.
theptip 4y ago
It’s one of those systemic health type things. It’s really hard to die of tech debt on its own, but if you move slower, you’ll die more often from other shocks. Another way of thinking about it is that you have N months of runway, and based on your velocity you can pull off a pivot in M months, and the more tech debt, the more time it will take to successfully pivot. If you don’t have a full pivot worth of runway remaining, and you need to pivot, you die. (Of course this oversimplifies by holding pivot magnitude equal but hopefully this illustrates the point.) I do agree that away from the margin, companies that are incredibly successful can afford to punt harder on tech debt. I suppose “know thyself” might be useful advice here; it’s probably not good advice for the median startup to ignore tech debt completely IMO. I think the main point though is to optimize for agility; tech debt can let you move faster in the short and even medium term, so sometimes it’s right to tactically take on some debt. But not so much that you get bogged down later; make sure you carve out time to fix the stuff that starts to be painful.
nonameiguess 4y ago
I feel like the biggest problem from a business strategy perspective is all we have are these personal opinions and gut feels. Even this article mentions having done 20 code audits, but presents nothing but qualitative findings. Ideally, some business school out there would be embedding researchers in randomly selected startups to know for sure how often you fail because of tech debt versus failing because of worrying too much about tech debt. That's an empirical question, yet all we get are informed expert opinions, but no auditable, reproducible research evidence. It's all so unscientific. Not to say you're wrong, but we have no real way of even deciding. All I can do is lean on my own experience, but I've seen nowhere near every product team out there and the ones I have seen are nothing close to randomly sampled or blinded.

spdionis 4y ago

Well, why are those companies pivoting so often in the first place? Isn't the root cause probably in GP's list?

phphphphp 4y ago

I’m not sure I agree. Technical debt is a symptom, it’s the consequence of bad management that leads to working on the wrong things. If you’re running a startup and haven’t yet found your feet in terms of a product offering, and you’re building your product(s) in such a way that technical debt builds up through continuously layering half-baked on half-baked, it’s indicative that you’re not actually pivoting and not actually evolving, you’re just adding new half-baked ideas to a half-baked system… and being able to do that at twice the speed isn’t going to address the real problem: half-baked ideas don’t make a product, whether that’s 10 half-baked ideas or 100. My experience is that any company in which evolution/experiments/pivoting is constrained within the boundaries of what already exists because of the sunk cost fallacy has made a grave error at a leadership level, not at a code level. If you can’t validate something without mashing more code into your system, that’s the problem to address. I’ve seen companies with horrendous tech debt die, and you could certainly frame their death as being a consequence of the tech debt (“if they had just got the perfect system…”) but that assumes the perfect system would somehow prevent them from making the mistakes that got them there in the first place. It wouldn’t. The technical debt is an expression of their mistakes, not the cause. You could dump the perfect system at their feet and they’d be surrounded by garbage again a few years from now.

icedchai 4y ago
I worked at a company that was mired in tech debt. At least 4 different UI frameworks were in use, one of which was totally not supported. Multiple versions of the app were left accessible, with links from the new to the old, because the new version was not feature complete. "Feature flags" were expressed in at least 3 different ways. It was a nightmare to figure out if something was on or off, and why. The back end was based on a non supported language version, with several old, deprecated third-party packages as a result. The company appeared organized, superficially, but at the lower levels of implementation it was a total dumpster fire. They were constantly "pivoting", but leaving the old junk around.
- redisman 4y ago
  There’s tech debt and then there’s poor engineering leadership. There’s no valid reason for a startup looking for market fit to switch frameworks or feature flags multiple times unless you’re just being clueless and looking for silver bullets. Just pick a few “boring” technologies and you’ll be perfectly capable of building anything “web” for at least a decade without messing around .

new_stranger 4y ago

And keeping 100% of all features instead of removing the least-used features as you add new ones to keep tech debt from growing indefinitely and reaching a point where new features take months to ship.

JamesBarney 4y ago

What killed them was they never found PMF. Eventually the tech debt slowed them down so they couldn't take as many swings at finding PMF. But in the counterfactual if they'd tried really hard to avoid tech debt that would have slowed them down at the beginning, not to mention there are plenty of organizations that will write very complex abstract code to avoid tech debt, and end up making the code base incredibly painful to work with. So overall did get they get less swings? I've worked on a lot of old code bases and the biggest issues I've run into, issues that crippled development velocity, were 95% boneheaded decisions and overengineering. And never the types of code quality issues someone like Uncle Bob talks about in Clean Code.