4 ms·

Embrace technical debt

68 points by nreece 17y ago

wynand 17y ago
This post makes a good companion to a post of a couple of days ago: "Why Do We Keep Building Tightly Coupled Software" (http://news.ycombinator.com/item?id=705863 http://news.ycombinator.com/item?id=705863). A common theme in the comments to that story was that there are costs associated with loose coupling. Not making loose coupling a first priority means that one will incur some technical debt; but given that one gets something that will work sooner, this is often a good call. Does anyone know of any software teams that explicitly talk about their technical debts and that have plans for "downpayments"?
- sokoloff 17y ago
  We do. Our [internal] wiki article on technical debt explicitly covers examples, guidance on when to intentionally take on debt, how to document and measure it, different "grades" of it, and how to plan to retire it. There are then per-application technical debt pages for specific details on our "portfolio" of debt. (We're also a fair bit larger than most startup dev teams, so having this level of formalism is more appropriate in our case than say just a whiteboard somewhere...)
  - darkxanthos 17y ago
    I would love to know your formal approach (or as close to it as possible). I personally see technical debt as debt to be leverage but I lack the means to track it so that we are always aware of how much debt we have.
    - sokoloff 17y ago
      We (or at least I) also think of it as leverage. We also have a very strong 80/20 ethic, so we don't have particular qualms about using the leverage that this debt can provide. We don't do anything magical; there is no secret sauce, noe no spreadsheets with formulae or anything fancy; it's literally just a series of wiki articles and Jira tickets with issue type "Debt" so we can easily find the documented debt and make plans when/whether to retire it. So the concrete formal mechanism is "use your bug tracking system and flag them somehow so you can filter/search for them." I will add some additional links that our top-level debt page references as useful: http://chrissterling.gettingagile.com/2008/10/20/managing-software-debt-article/ http://chrissterling.gettingagile.com/2008/10/20/managing-so... http://enterprisearchitect.typepad.com/ea/2008/10/watch-your-amou.html http://enterprisearchitect.typepad.com/ea/2008/10/watch-your... http://www.youtube.com/watch?v=pqeJFYwnkjE http://www.youtube.com/watch?v=pqeJFYwnkjE (Video: Ward Cunningham on technical debt)
kreem 17y ago
I think this is a good article, but most times companies do not even know they are incurring technical debt, and that is the first step before you can start working with technical debt in an intelligent way.
jasongullickson 17y ago
Weird, I was just trying to explain this to a manager yesterday, I need to go look for hidden microphones... ...I've tried in the past to explain this to others using retail "shrink" as an analogy. For those unfamiliar with the term, as I know it, the idea is that you assume a certain percentage of loss (in my experience, as shoplifting in a music store) and by writing that off, it allows you to focus your energies where they have more impact on profit. For example, placing the lowest-margin items in the most vulnerable parts of the store to deter theft of items where you really make money. I'm glad someone took the time to write up a better explanation that I can now refer to.
- lallysingh 17y ago
  LiveScribe pens.
deleted 17y ago
[deleted]
jon_dahl 17y ago
I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow". I've worked on too many projects that raced to cram in features by a deadline, and ended up with a sloppy app that took forever to maintain and made developers miserable. Far better to take a measured pace and ensure that rapid development can happen in months 12-24, not just months 1-3. Especially in startups - startups are a marathon, not a sprint. Of course, cutting corners isn't the only kind of code debt, and the author makes some good points. If you're spiking something or prototyping it, and intend to rewrite or throw away, technical debt might be a good tradeoff. I also take issue with this: Early on at IMVU, we incorporated in tons of open source projects. This was a huge win... The downside was that we had to combine dozens of projects whose internal architectures, coding styles, and general quality varied widely. It took us a long time to pay off all the debt that incurred – but it was worth it. The more you can offload to open source libraries or third party APIs, the less your application has to do, and the less you have to maintain. If the library or API sucks, then sure - you're incurring technical debt. But if the library/API is good, or you can contribute to make it good, then it's a win all around.
- swombat 17y ago
  I also take issue with this: Early on at IMVU, we incorporated in tons of open source projects. This was a huge win... The downside was that we had to combine dozens of projects whose internal architectures, coding styles, and general quality varied widely. It took us a long time to pay off all the debt that incurred – but it was worth it. I think that's more or less exactly the point he makes. Some external dependencies are good, some are dreadful. At some point, you need to address the bad ones. I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow". Re: cutting corners, it really depends what you mean by "cutting corners". There's a whole continuum of code quality from "omg this sucks I want to kill the person who wrote it" to "this is the most beautiful piece of code every designed by man. Anything short of the latter is effectively incurring debt. I don't think Eric Ries is advocating dirty, nasty coding. Instead, I'd say he's suggesting that you should be comfortable with not writing the best possible code, rather than always feel like you're not doing well enough.
  - 17y ago
apinstein 17y ago
Technical Debt is a broken metaphor. It's almost right, which is why it resonates, but it's a bad mental model for the problem it tries to describe. There are real negative consequence of applying a "broken mental model" -- you end up doing the wrong thing when you live by the model! Why is technical debt a broken concept? Real Debt is incurred proportionally to the asset bought with it. In technical debt the debt is proportional to the trouble it causes, not the value delivered. Also, it implies that the only way to get rid of the debt is to make a big investment. In reality, you can just throw away code and the debt disappears. Not analogous with real debt! It should've been called technical drag. Properties of drag that are appropriate to the metaphor: - You get value based on the amount of input work, but it places a drag on the overall progress towards the goal (support load, architectural coupling). - You can reduce the drag of an existing component by investing to refine it. - New components typically add more drag than existing ones. - Little bits of drag here and there add up to a big problem - Removing unused functionality reduces associated drag without major investment Thoughts?
- swombat 17y ago
  Agree. In fact, I called it "viscosity" in an article a long time ago, and posited that the rate of increase of viscosity depends on languages, practices, etc: http://inter-sections.net/2007/12/01/comparing-pieces-of-string-part-1 http://inter-sections.net/2007/12/01/comparing-pieces-of-str... "Technical drag" is a very good term for it, I agree.
  - apinstein 17y ago
    Interesting article. I think using viscosity to compare languages or frameworks is counterproductive, but the general idea is useful when talking about software architecture and development. For instance, I used to manage some poorly architected Java apps where even trivial changes incurred painful re-architecting because the app was poorly written. And I've managed PHP apps that are extremely well-organized and are very simple to add functionality to. The other thing I'd mention is I like your idea about hitting the wall where improvement freezes b/c the implementation time is so great. I think it's a good fit for the technical drag model -- that is the point of terminal velocity! It's another good-fit for the metaphor; eventually you can only drive forward a product by making the team bigger because your project hits its terminal velocity (or top speed) for the given "engine" size (the team).
wglb 17y ago
This is a good article and it resonates with one of pg's essays in which he says that sometimes in building a project you worry about security later. I think it is also related to the early optimization error. You don't know what you are ultimately going to build until you get feedback from customers/users. So don't overengineer/optimize something that you aren't going to use/doesn't need it.
jimbokun 17y ago
"As in many scalability decisions, we’d have been much better off investing in agility, so that we could change the architecture in response to actual customer demand, rather than trying to predict the future." With a poor design, it can be more difficult to make changes to improve scalability. If you do not have abstractions and information hiding that allow you to switch out algorithms, libraries or systems for ones that scale better, you will need to rewrite a lot more code in order to to scale. "Yet other agile principles suggest the opposite, as in YAGNI and DoTheSimplestThingThatCouldPossiblyWork." My concern with the premise of this article is that some might use it as a license for sloppy code. YAGNI and DoTheSimplestThingThatCouldPossiblyWork can be accomplished with code that is concise and clearly expresses intent. In fact, you could argue that these principles demand such code. I've seen a lot of code with many lines that accomplish very little, and that kind of "technical debt" never pays off, in my experience.

68 points by nreece 17y ago

This post makes a good companion to a post of a couple of days ago: "Why Do We Keep Building Tightly Coupled Software" (http://news.ycombinator.com/item?id=705863 http://news.ycombinator.com/item?id=705863). A common theme in the comments to that story was that there are costs associated with loose coupling. Not making loose coupling a first priority means that one will incur some technical debt; but given that one gets something that will work sooner, this is often a good call. Does anyone know of any software teams that explicitly talk about their technical debts and that have plans for "downpayments"?

sokoloff 17y ago
We do. Our [internal] wiki article on technical debt explicitly covers examples, guidance on when to intentionally take on debt, how to document and measure it, different "grades" of it, and how to plan to retire it. There are then per-application technical debt pages for specific details on our "portfolio" of debt. (We're also a fair bit larger than most startup dev teams, so having this level of formalism is more appropriate in our case than say just a whiteboard somewhere...)
- darkxanthos 17y ago
  I would love to know your formal approach (or as close to it as possible). I personally see technical debt as debt to be leverage but I lack the means to track it so that we are always aware of how much debt we have.
  - sokoloff 17y ago
    We (or at least I) also think of it as leverage. We also have a very strong 80/20 ethic, so we don't have particular qualms about using the leverage that this debt can provide. We don't do anything magical; there is no secret sauce, noe no spreadsheets with formulae or anything fancy; it's literally just a series of wiki articles and Jira tickets with issue type "Debt" so we can easily find the documented debt and make plans when/whether to retire it. So the concrete formal mechanism is "use your bug tracking system and flag them somehow so you can filter/search for them." I will add some additional links that our top-level debt page references as useful: http://chrissterling.gettingagile.com/2008/10/20/managing-software-debt-article/ http://chrissterling.gettingagile.com/2008/10/20/managing-so... http://enterprisearchitect.typepad.com/ea/2008/10/watch-your-amou.html http://enterprisearchitect.typepad.com/ea/2008/10/watch-your... http://www.youtube.com/watch?v=pqeJFYwnkjE http://www.youtube.com/watch?v=pqeJFYwnkjE (Video: Ward Cunningham on technical debt)

kreem 17y ago

I think this is a good article, but most times companies do not even know they are incurring technical debt, and that is the first step before you can start working with technical debt in an intelligent way.

jasongullickson 17y ago

Weird, I was just trying to explain this to a manager yesterday, I need to go look for hidden microphones... ...I've tried in the past to explain this to others using retail "shrink" as an analogy. For those unfamiliar with the term, as I know it, the idea is that you assume a certain percentage of loss (in my experience, as shoplifting in a music store) and by writing that off, it allows you to focus your energies where they have more impact on profit. For example, placing the lowest-margin items in the most vulnerable parts of the store to deter theft of items where you really make money. I'm glad someone took the time to write up a better explanation that I can now refer to.

lallysingh 17y ago
LiveScribe pens.

deleted 17y ago

[deleted]

jon_dahl 17y ago

I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow". I've worked on too many projects that raced to cram in features by a deadline, and ended up with a sloppy app that took forever to maintain and made developers miserable. Far better to take a measured pace and ensure that rapid development can happen in months 12-24, not just months 1-3. Especially in startups - startups are a marathon, not a sprint. Of course, cutting corners isn't the only kind of code debt, and the author makes some good points. If you're spiking something or prototyping it, and intend to rewrite or throw away, technical debt might be a good tradeoff. I also take issue with this: Early on at IMVU, we incorporated in tons of open source projects. This was a huge win... The downside was that we had to combine dozens of projects whose internal architectures, coding styles, and general quality varied widely. It took us a long time to pay off all the debt that incurred – but it was worth it. The more you can offload to open source libraries or third party APIs, the less your application has to do, and the less you have to maintain. If the library or API sucks, then sure - you're incurring technical debt. But if the library/API is good, or you can contribute to make it good, then it's a win all around.

swombat 17y ago
I also take issue with this: Early on at IMVU, we incorporated in tons of open source projects. This was a huge win... The downside was that we had to combine dozens of projects whose internal architectures, coding styles, and general quality varied widely. It took us a long time to pay off all the debt that incurred – but it was worth it. I think that's more or less exactly the point he makes. Some external dependencies are good, some are dreadful. At some point, you need to address the bad ones. I tell my clients that cutting corners is "an hour of savings today, 5 hours of cost tomorrow". Re: cutting corners, it really depends what you mean by "cutting corners". There's a whole continuum of code quality from "omg this sucks I want to kill the person who wrote it" to "this is the most beautiful piece of code every designed by man. Anything short of the latter is effectively incurring debt. I don't think Eric Ries is advocating dirty, nasty coding. Instead, I'd say he's suggesting that you should be comfortable with not writing the best possible code, rather than always feel like you're not doing well enough.
- 17y ago

apinstein 17y ago

Technical Debt is a broken metaphor. It's almost right, which is why it resonates, but it's a bad mental model for the problem it tries to describe. There are real negative consequence of applying a "broken mental model" -- you end up doing the wrong thing when you live by the model! Why is technical debt a broken concept? Real Debt is incurred proportionally to the asset bought with it. In technical debt the debt is proportional to the trouble it causes, not the value delivered. Also, it implies that the only way to get rid of the debt is to make a big investment. In reality, you can just throw away code and the debt disappears. Not analogous with real debt! It should've been called technical drag. Properties of drag that are appropriate to the metaphor: - You get value based on the amount of input work, but it places a drag on the overall progress towards the goal (support load, architectural coupling). - You can reduce the drag of an existing component by investing to refine it. - New components typically add more drag than existing ones. - Little bits of drag here and there add up to a big problem - Removing unused functionality reduces associated drag without major investment Thoughts?

swombat 17y ago
Agree. In fact, I called it "viscosity" in an article a long time ago, and posited that the rate of increase of viscosity depends on languages, practices, etc: http://inter-sections.net/2007/12/01/comparing-pieces-of-string-part-1 http://inter-sections.net/2007/12/01/comparing-pieces-of-str... "Technical drag" is a very good term for it, I agree.
- apinstein 17y ago
  Interesting article. I think using viscosity to compare languages or frameworks is counterproductive, but the general idea is useful when talking about software architecture and development. For instance, I used to manage some poorly architected Java apps where even trivial changes incurred painful re-architecting because the app was poorly written. And I've managed PHP apps that are extremely well-organized and are very simple to add functionality to. The other thing I'd mention is I like your idea about hitting the wall where improvement freezes b/c the implementation time is so great. I think it's a good fit for the technical drag model -- that is the point of terminal velocity! It's another good-fit for the metaphor; eventually you can only drive forward a product by making the team bigger because your project hits its terminal velocity (or top speed) for the given "engine" size (the team).

wglb 17y ago

This is a good article and it resonates with one of pg's essays in which he says that sometimes in building a project you worry about security later. I think it is also related to the early optimization error. You don't know what you are ultimately going to build until you get feedback from customers/users. So don't overengineer/optimize something that you aren't going to use/doesn't need it.

jimbokun 17y ago

"As in many scalability decisions, we’d have been much better off investing in agility, so that we could change the architecture in response to actual customer demand, rather than trying to predict the future." With a poor design, it can be more difficult to make changes to improve scalability. If you do not have abstractions and information hiding that allow you to switch out algorithms, libraries or systems for ones that scale better, you will need to rewrite a lot more code in order to to scale. "Yet other agile principles suggest the opposite, as in YAGNI and DoTheSimplestThingThatCouldPossiblyWork." My concern with the premise of this article is that some might use it as a license for sloppy code. YAGNI and DoTheSimplestThingThatCouldPossiblyWork can be accomplished with code that is concise and clearly expresses intent. In fact, you could argue that these principles demand such code. I've seen a lot of code with many lines that accomplish very little, and that kind of "technical debt" never pays off, in my experience.