9 ms·

Garbage collect your technical debt (2021)

136 points by gfairbanks 2y ago

Rochus 2y ago
It even rhymes ;-)
- oriel 2y ago
  Schoolhouse Rock for Coders?
gfairbanks 2y ago
A quirk of IEEE's publishing system is that it drops the abstract, instead using the first paragraph. Here is the abstract [1]: The iterative process that a team follows is a bit like a garbage collection algorithm, and we can compare software development processes like we can any algorithm. A process can help developers do two things: clean up tech debt after it exists, or avoid creating it. When an iterative process does neither, tech debt buildup will lead to bankruptcy, so it is only suitable for projects with a short lifespan. A process that does both has the best chance at minimizing tech debt over a long lifespan. In particular, focusing on the system’s design will keep tech debt low. [1] https://www.georgefairbanks.com/ieee-software-v38-n5-sep-2021-garbage-collect-your-tech-debt https://www.georgefairbanks.com/ieee-software-v38-n5-sep-202...
- deleted 2y ago
  [deleted]
clcaev 2y ago
It's not always easy to find the right words to describe why a feature pause to refactor is needed. This quote was a succinct statement that avoids analogy: "Postponing a small cleanup can transform it into a big cleanup because, over time, code builds up around the problem, and it too must be refactored."
- WalterSear 2y ago
  "I'm sorry, I didn't understand much of that. Are you saying you can commit to finishing the feature on a shorter timeframe than you originally asked for? Would it help if we forgo writing tests?"
  - nxicvyvy 2y ago
    Yes, but every other feature will take ten times as long as over the next three years we will lose 90% of our best developers.
    - RadiozRadioz 2y ago
      "This is the last feature we will tell you to rush, we promise. We really need just this one, and then we're good."
      - deleted 2y ago
        [deleted]
      - FridgeSeal 2y ago
        “Well done on delivering everyone! We knew you could do it, now, our next priority is this new thing……”
        kaffekaka 2y ago
        Please stop, it hurts.
      - Seattle3503 2y ago
        Says the PM, right before they get promoted.
    - WalterSear 2y ago
      "I'm confused. When I greenfielded this app, several thousand commits ago, I took half the time you've already spent on this feature. You said you were a senior engineer!!" True story. Twice.
notjoemama 2y ago
> pushed a routine re-acceptance of those terms Oh, of course, of course. Why, this is so mundane, why would anyone even bother to read it? > For content processed or stored on Adobe servers, Adobe may use technologies and other processes, including escalation for manual (human) review, to screen for certain types of illegal content (such as child sexual abuse material), or other abusive content or behavior (for example, patterns of activity that indicate spam or phishing). They are going to use AI to scan through all content...for your protection. Does Adobe have a problem with people using their cloud to produce child abuse material? I think we all recognize Bitcoin is used for illegal activity. But I never considered Adobe being associated with CP. I guess that's what they're going with... I feel like this is a trope at this point. Just like Microsoft Recall, no one believes or trusts this won't eventually turn into them using your IP for themselves via AI, and then eventually by the government. That's the bottom line. To really put a point on it, they are saying they're swipping a spider off us while they slip their hands down our pants.
- rrdharan 2y ago
  commented on the wrong post?
caseyohara 2y ago
> Building software iteratively leads inevitably to tech debt because we choose to deliver systems before we have looked at all the requirements. Not knowing what’s next distorts our designs, and that distortion is the tech debt. This article frames technical debt as something that happens passively because you can't know future requirements. That's sometimes true, of course, but in my experience the majority of technical debt is accrued deliberately in a much more active process. When developing a new feature that doesn't neatly fit into the existing system, you must choose between two compromises: 1. Build it the "fast way", shoehorning the feature into the system just enough to work, compromising quality for velocity and accruing technical debt; or 2. Build it the "right way", adapting the system to accommodate the new feature, compromising velocity for quality to avoid technical debt. This is usually a deliberate decision, so choosing to accrue technical debt is an active process. The only way it could be passive like the article describes is if the developers don't know or otherwise don't consider the "right way" and go straight for the "fast way". I hope to never work on a team that operates like that.
- dec0dedab0de 2y ago
  The problem is that there is never an objective right way. There are infinite wrong ways, and usually a handful of ways that are just fine with different pros and cons that are not always clear up front. A lot of the time when people say technical debt they mean a developer not taking the time to understand some code they inherited (or they wrote and forgot about), and wanting to throw the baby out with the bathwater. If think you ask ten developers the best way to refactor a complex program you'll get 100 answers. But I do agree that deliberate technical debt is more common on a decent team. I definitely have left many comments like "I know it would be better if I did XYZ, or ABC, but the boss wants it now, and I'm tired, so it's going to be a monstrosity"
  - yarekt 2y ago
    the 10 -> 100 bit is absolutely right. a cross functional team has to learn how to work together well, and spend a non trivial amount of time discussing the problem and tempering solutions until something is achieved where each of them are some state of “happy”. This is I think the crux of many tech debt issues in companies
gumby 2y ago
Incremental collection is preferred in this case, as stop-and-copy collection leads to Second System Syndrome.
digger495 2y ago
This is one of the dumbest and most obvious journal articles I've ever read.
bitwize 2y ago
RAII your technical debt: make a note of every time you slap something together the expedient way, and schedule a future time to refactor after the legitimacy of expediency goes "out of scope", e.g., after a major product demo. One could even add a CI/CD task to remind the user of when refactorings are due (and refuse to successfully build until they are addressed). Since debt is created by borrowing, one could call this the "borrow checker".
- fiddlerwoaroof 2y ago
  A couple times I've put a "if (now() > xxxxxx) fail()` in tests for this reason.
  - rglynn 2y ago
    How did you get that past peer review?
    - fiddlerwoaroof 2y ago
      Because people agreed that the failing test represented technical debt that needed to be cleaned up later.
  - yarekt 2y ago
    Good in theory, but try that on a project with real customers and you’ll get a very simple “remove it and we’re done here” /shrug. Team’s processes and decision chain must have agreed to enforce this already, and that above is just a symptom
  - ralferoo 2y ago
    I think this is the absolute worst way of dealing with the issue, and I truly hope everyone who sees this in a code-review has the sense to reject it. By all means, have such a check as a compile-time error, but not as something that gets shipped to customers. The code you have now works, even if as a developer you'd like to refactor it. However, what if you ship this software to a paying customer, and then your company goes bust? At unknown point in the future, their perfectly good application suddenly stops working, without warning or obvious cause, and your company is no longer in business to simply change the time to a later one and recompile. Even if your company hasn't gone bust, why should the customer have to spend ages trying to figure out if they've done something wrong or it's the software that's at fault, before contacting your company, possibly being told to pay more money for an update even if they were otherwise happy with the old version they'd purchased. Basically, this change is a liability for everybody working on the project and your customers. It'll soon get forgotten about, until you have people complaining that something just suddenly stopped working without warning. You might be able to "fix" it in seconds, but it can still require hours to triage the issue to discover what caused it before it ends up as your problem again, and probably significantly longer after your few seconds to fix it before the working version is back in the hands of the customer. For all that time, they've been inconvenienced just to save you the hassle of sticking a reminder in your own personal calendar or raising an issue in your bug tracking system or wherever else.
m463 2y ago
realtime systems just prealloc everything at the beginnging.
noisy_boy 2y ago
I do opportunistic collection - when I am working on a feature, and spot opportunities for a refactoring/cleanup in code that is more or less directly related to the code I'm touching, I will keep making small incremental changes and keep testing them until my feature is implemented and the cleanup is done as well. I also ensure to not make a breaking change while doing this e.g. no change to the user facing api signature. If I see issues in unrelated code, that just gets a TODO and waits its turn when we have to do changes there for some feature request. The reason is that I can atleast somewhat justify the changes under the umbrella of my feature while utilizing the allocated time budget. If I don't do this, we will never get a dedicated release to do tech cleanup - the backlog of feature requests is just too big and too little appetite on the decision makers' side for purely tech debt releases.
- TeeMassive 2y ago
  This is the only way I've found that we can do the needed refactor. As consultants, we never ask the permission of the client to refactor the code. Ever. That's not for them to decide. The reason they hired us in the first place is because we're the experts and they are not. We evaluate the needed refactor that puts the code in a decent state and put the time in the quote we provide the client. And even then we never create tasks or log time; when we find something that needs to be done, then our time is billed to the task relating to that issue.
- teeray 2y ago
  I have done this in the past and it has worked well. On some teams though, I have been met with: “Why did you do this refactor with the feature? Can you pull the feature into another PR, then we’ll leave the refactor in the original PR to be merged at another time? (read: never)”
  - ralferoo 2y ago
    I'd definitely agree with the putting the refactor into a separate PR and then the new functionality in another. Aside from the obvious that people are more likely to be willing to review your change when it's small, it makes it much easier reason about both when they are separate. There will be a lot of noise generated from the refactor that will drown out the new feature, but self-contained in its own PR it's a lot easier to understand the new feature and spot mistakes. A simple refactor can likewise be likely skimmed over quickly, looking for the patterns of how it was done and assuming that most of the changes were similar and mostly just looking out for the differences. As for accepting the feature PR but not the refactor PR, that suggests that the feature doesn't actually rely on the refactor. In that case, it's even clearer that they should be separated. Personally, I'd always even create a new ticket for the refactor so that there's some justification for the work, and maybe you can say that this ticket blocks the one for the actual work. Maybe that's just me though, because I like to make sure that every non-trivial commit is tied to an issue in the bug/issue tracking software. This means that every bit of code is a simple "git blame" away from a justification of why it was changed.
marcus_holmes 2y ago
So, from my 30+years building software products, I have a couple of problems with this: - we don't choose iterative development as an alternative to waterfall because "waterfall is bad mmkay". When developing a new product, we don't know all the requirements. Part of the process of iterative development is discovering the requirements. Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated (even <especially> when every stakeholder pinky-swore that the requirements were final before Phase 1 started). - The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users. - Changing the design to accommodate the new feature is great in theory, but in a lot of cases the new feature is an experiment itself. We're optimising for customer learning: get the new feature in quickly with the minimum of effort, see if the users actually use it, and if not then roll back. If the users do like it, then refactor to tidy up any tech debt. Doing all the refactoring first is a complete waste of time if the feature ends up getting rolled back. - Tech debt slows down development but only as a result of delivering early features quicker. It's the sharpening the axe analogy - sooner or later you have to stop chopping the tree to sharpen the axe. Changing the way you chop the tree so you don't have to sharpen the axe is not optimising for getting the most timber quickest. The trick here is communicating with stakeholders about what tech debt is and how it accumulates. They're usually OK with feature pauses if they understand the situation correctly. In other words, like a lot of problems in this industry, this is a communication problem not actually a tech problem.
- naasking 2y ago
  > Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated This is a false equivalence. Waterfall is iterative in requirements but not in code. Changing the requirements document is much, much simpler than changing the code to conform to new requirements. > The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users. Not sure I agree with the implication "learning from users entails accruing technical debt". Technical debt is a rough measure of the inherent resistance of your program to adapt to customer requirements. Some architectures are flexible enough that you could learn how to configure the architecture differently based on what you've learned from users, but I don't see how that necessarily entails the addition of technical debt, ie. as a rule.
woodpanel 2y ago
Ctrl+F "@TODO"… or have your build pipeline keep track of Todos automatically. Also some IDE integrations keep track and provide quicklinks to lists of TODOs.
- yarekt 2y ago
  Curious. Got any recommendations for CI tooling that analyses todos? What sort of things that you’ve seen are possible
  - shepherdjerred 2y ago
    I read this interesting paper a year or two back about triggerable TODOs in Java: https://users.ece.utexas.edu/~gligoric/papers/NieETAL19TrigIt.pdf https://users.ece.utexas.edu/~gligoric/papers/NieETAL19TrigI...
MeteorMarc 2y ago
The garbage collector already has a 100% coverage test suite.
anymouse123456 2y ago
Wow, what a weird feeling. The authors of this article seem like super smart and experienced guys, but the article itself is introduced with incorrect statements in the very first two sentences and slides downhill from there. "There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all of the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions." 1) There isn't an option to build software by "looking at all of the requirements at once." The requirements of any (sufficiently complex) project will emerge over the course of development, regardless of whether it's built in an iterative style, or with a much larger investment in planning and design up front. We often work iteratively because we acknowledge this particular aspect of reality and it helps everyone when we work closer to reality, rather than fighting it. 2) Technical Debt does not describe design distortions that arise due to "iterative" development, it describes design problems that arise in every project. Quantizing the choices for dealing with tech debt into 4 buckets doesn't feel right either. Changing the design of a running system happens along a continuum and the need and benefit vary widely over the course of any given software lifecycle and surrounding (business or other) environment. Maybe I'm just grumpy this morning, and I think reasonable folks could disagree, but the metaphor at the center of the thesis (tech debt ≅ soon-to-be-deallocated-memory) doesn't hold up for me at all. Tech debt is sometimes left in place, intentionally or not, for many years. Sometimes we chip away at it, sometimes we stop the world and push on it. Sometimes it's bad enough to throw the whole system away and start over. It's like debt for businesses. It can be valuable to take on some debt if it lets you stay alive long enough to pay it back. Finally, I'm struggling with this article because tech debt is already a metaphor, that includes information about how, and when to take it on and pay it down. It's not helpful to layer another, unrelated metaphor on top of it.
- gfairbanks 2y ago
  Ward Cunningham's original idea of tech debt (see [1], a beauty of concision at just 300 words) is that iterative development distorts your code because you start writing code before you know the requirements, but even so, it's better than waterfall. "The traditional waterfall development cycle has endeavored to avoid programming catastrophy by working out a program in detail before programming begins. We ... [instead use] the alternative, incremental growth ..." Today, the term "tech debt" includes sloppy code, shortcuts, novice code -- really any kind of bad code. The original conception of tech debt is more limited and tied to waterfall vs iterative process choices. [2] [1] Ward Cunningham, The WyCash Portfolio Management System, OOPSLA 1992. https://c2.com/doc/oopsla92.html https://c2.com/doc/oopsla92.html [2] George Fairbanks, Ur-Technical Debt, IEEE Software 2020. https://ieeexplore.ieee.org/document/9121630 https://ieeexplore.ieee.org/document/9121630
igornadj 2y ago
Does anyone else get the feeling that how we deal with technical debt in software is fundamentally and critically misguided? How a pot hole gets filled in isn't a technical challenge, its a political challenge. Developers can talk about the _technical_ side of technical debt all day long, but its a) generally well understood already, and b) blowing in the wind. How technical debt is handled is fundamentally a political problem. We may understand this at a glance, but the solution of "gain knowledge and bring it up at standup" is not only not getting us anywhere, it's deliberately misleading us into thinking its up to us individually to change the organisation. It's as silly as telling the labourer that part of their job is to lobby the city council to spend more time and resources on filling pot holes. Discussions on technical debt should be within the bounds of how to identify organisations where technical debt is identified as a real issue from the top down, and a survival guide for working in organisations where it is not.

136 points by gfairbanks 2y ago

Rochus 2y ago

It even rhymes ;-)

oriel 2y ago
Schoolhouse Rock for Coders?

gfairbanks 2y ago

A quirk of IEEE's publishing system is that it drops the abstract, instead using the first paragraph. Here is the abstract [1]: The iterative process that a team follows is a bit like a garbage collection algorithm, and we can compare software development processes like we can any algorithm. A process can help developers do two things: clean up tech debt after it exists, or avoid creating it. When an iterative process does neither, tech debt buildup will lead to bankruptcy, so it is only suitable for projects with a short lifespan. A process that does both has the best chance at minimizing tech debt over a long lifespan. In particular, focusing on the system’s design will keep tech debt low. [1] https://www.georgefairbanks.com/ieee-software-v38-n5-sep-2021-garbage-collect-your-tech-debt https://www.georgefairbanks.com/ieee-software-v38-n5-sep-202...

deleted 2y ago
[deleted]

clcaev 2y ago

It's not always easy to find the right words to describe why a feature pause to refactor is needed. This quote was a succinct statement that avoids analogy: "Postponing a small cleanup can transform it into a big cleanup because, over time, code builds up around the problem, and it too must be refactored."

WalterSear 2y ago
"I'm sorry, I didn't understand much of that. Are you saying you can commit to finishing the feature on a shorter timeframe than you originally asked for? Would it help if we forgo writing tests?"
- nxicvyvy 2y ago
  Yes, but every other feature will take ten times as long as over the next three years we will lose 90% of our best developers.
  - RadiozRadioz 2y ago
    "This is the last feature we will tell you to rush, we promise. We really need just this one, and then we're good."
    - deleted 2y ago
      [deleted]
    - FridgeSeal 2y ago
      “Well done on delivering everyone! We knew you could do it, now, our next priority is this new thing……”
      - kaffekaka 2y ago
        Please stop, it hurts.
    - Seattle3503 2y ago
      Says the PM, right before they get promoted.
  - WalterSear 2y ago
    "I'm confused. When I greenfielded this app, several thousand commits ago, I took half the time you've already spent on this feature. You said you were a senior engineer!!" True story. Twice.

notjoemama 2y ago

> pushed a routine re-acceptance of those terms Oh, of course, of course. Why, this is so mundane, why would anyone even bother to read it? > For content processed or stored on Adobe servers, Adobe may use technologies and other processes, including escalation for manual (human) review, to screen for certain types of illegal content (such as child sexual abuse material), or other abusive content or behavior (for example, patterns of activity that indicate spam or phishing). They are going to use AI to scan through all content...for your protection. Does Adobe have a problem with people using their cloud to produce child abuse material? I think we all recognize Bitcoin is used for illegal activity. But I never considered Adobe being associated with CP. I guess that's what they're going with... I feel like this is a trope at this point. Just like Microsoft Recall, no one believes or trusts this won't eventually turn into them using your IP for themselves via AI, and then eventually by the government. That's the bottom line. To really put a point on it, they are saying they're swipping a spider off us while they slip their hands down our pants.

rrdharan 2y ago
commented on the wrong post?

caseyohara 2y ago

> Building software iteratively leads inevitably to tech debt because we choose to deliver systems before we have looked at all the requirements. Not knowing what’s next distorts our designs, and that distortion is the tech debt. This article frames technical debt as something that happens passively because you can't know future requirements. That's sometimes true, of course, but in my experience the majority of technical debt is accrued deliberately in a much more active process. When developing a new feature that doesn't neatly fit into the existing system, you must choose between two compromises: 1. Build it the "fast way", shoehorning the feature into the system just enough to work, compromising quality for velocity and accruing technical debt; or 2. Build it the "right way", adapting the system to accommodate the new feature, compromising velocity for quality to avoid technical debt. This is usually a deliberate decision, so choosing to accrue technical debt is an active process. The only way it could be passive like the article describes is if the developers don't know or otherwise don't consider the "right way" and go straight for the "fast way". I hope to never work on a team that operates like that.

dec0dedab0de 2y ago
The problem is that there is never an objective right way. There are infinite wrong ways, and usually a handful of ways that are just fine with different pros and cons that are not always clear up front. A lot of the time when people say technical debt they mean a developer not taking the time to understand some code they inherited (or they wrote and forgot about), and wanting to throw the baby out with the bathwater. If think you ask ten developers the best way to refactor a complex program you'll get 100 answers. But I do agree that deliberate technical debt is more common on a decent team. I definitely have left many comments like "I know it would be better if I did XYZ, or ABC, but the boss wants it now, and I'm tired, so it's going to be a monstrosity"
- yarekt 2y ago
  the 10 -> 100 bit is absolutely right. a cross functional team has to learn how to work together well, and spend a non trivial amount of time discussing the problem and tempering solutions until something is achieved where each of them are some state of “happy”. This is I think the crux of many tech debt issues in companies

gumby 2y ago

Incremental collection is preferred in this case, as stop-and-copy collection leads to Second System Syndrome.

digger495 2y ago

This is one of the dumbest and most obvious journal articles I've ever read.

bitwize 2y ago

RAII your technical debt: make a note of every time you slap something together the expedient way, and schedule a future time to refactor after the legitimacy of expediency goes "out of scope", e.g., after a major product demo. One could even add a CI/CD task to remind the user of when refactorings are due (and refuse to successfully build until they are addressed). Since debt is created by borrowing, one could call this the "borrow checker".

fiddlerwoaroof 2y ago
A couple times I've put a "if (now() > xxxxxx) fail()` in tests for this reason.
- rglynn 2y ago
  How did you get that past peer review?
  - fiddlerwoaroof 2y ago
    Because people agreed that the failing test represented technical debt that needed to be cleaned up later.
- yarekt 2y ago
  Good in theory, but try that on a project with real customers and you’ll get a very simple “remove it and we’re done here” /shrug. Team’s processes and decision chain must have agreed to enforce this already, and that above is just a symptom
- ralferoo 2y ago
  I think this is the absolute worst way of dealing with the issue, and I truly hope everyone who sees this in a code-review has the sense to reject it. By all means, have such a check as a compile-time error, but not as something that gets shipped to customers. The code you have now works, even if as a developer you'd like to refactor it. However, what if you ship this software to a paying customer, and then your company goes bust? At unknown point in the future, their perfectly good application suddenly stops working, without warning or obvious cause, and your company is no longer in business to simply change the time to a later one and recompile. Even if your company hasn't gone bust, why should the customer have to spend ages trying to figure out if they've done something wrong or it's the software that's at fault, before contacting your company, possibly being told to pay more money for an update even if they were otherwise happy with the old version they'd purchased. Basically, this change is a liability for everybody working on the project and your customers. It'll soon get forgotten about, until you have people complaining that something just suddenly stopped working without warning. You might be able to "fix" it in seconds, but it can still require hours to triage the issue to discover what caused it before it ends up as your problem again, and probably significantly longer after your few seconds to fix it before the working version is back in the hands of the customer. For all that time, they've been inconvenienced just to save you the hassle of sticking a reminder in your own personal calendar or raising an issue in your bug tracking system or wherever else.

m463 2y ago

realtime systems just prealloc everything at the beginnging.

noisy_boy 2y ago

I do opportunistic collection - when I am working on a feature, and spot opportunities for a refactoring/cleanup in code that is more or less directly related to the code I'm touching, I will keep making small incremental changes and keep testing them until my feature is implemented and the cleanup is done as well. I also ensure to not make a breaking change while doing this e.g. no change to the user facing api signature. If I see issues in unrelated code, that just gets a TODO and waits its turn when we have to do changes there for some feature request. The reason is that I can atleast somewhat justify the changes under the umbrella of my feature while utilizing the allocated time budget. If I don't do this, we will never get a dedicated release to do tech cleanup - the backlog of feature requests is just too big and too little appetite on the decision makers' side for purely tech debt releases.

TeeMassive 2y ago
This is the only way I've found that we can do the needed refactor. As consultants, we never ask the permission of the client to refactor the code. Ever. That's not for them to decide. The reason they hired us in the first place is because we're the experts and they are not. We evaluate the needed refactor that puts the code in a decent state and put the time in the quote we provide the client. And even then we never create tasks or log time; when we find something that needs to be done, then our time is billed to the task relating to that issue.
teeray 2y ago
I have done this in the past and it has worked well. On some teams though, I have been met with: “Why did you do this refactor with the feature? Can you pull the feature into another PR, then we’ll leave the refactor in the original PR to be merged at another time? (read: never)”
- ralferoo 2y ago
  I'd definitely agree with the putting the refactor into a separate PR and then the new functionality in another. Aside from the obvious that people are more likely to be willing to review your change when it's small, it makes it much easier reason about both when they are separate. There will be a lot of noise generated from the refactor that will drown out the new feature, but self-contained in its own PR it's a lot easier to understand the new feature and spot mistakes. A simple refactor can likewise be likely skimmed over quickly, looking for the patterns of how it was done and assuming that most of the changes were similar and mostly just looking out for the differences. As for accepting the feature PR but not the refactor PR, that suggests that the feature doesn't actually rely on the refactor. In that case, it's even clearer that they should be separated. Personally, I'd always even create a new ticket for the refactor so that there's some justification for the work, and maybe you can say that this ticket blocks the one for the actual work. Maybe that's just me though, because I like to make sure that every non-trivial commit is tied to an issue in the bug/issue tracking software. This means that every bit of code is a simple "git blame" away from a justification of why it was changed.

marcus_holmes 2y ago

So, from my 30+years building software products, I have a couple of problems with this: - we don't choose iterative development as an alternative to waterfall because "waterfall is bad mmkay". When developing a new product, we don't know all the requirements. Part of the process of iterative development is discovering the requirements. Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated (even <especially> when every stakeholder pinky-swore that the requirements were final before Phase 1 started). - The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users. - Changing the design to accommodate the new feature is great in theory, but in a lot of cases the new feature is an experiment itself. We're optimising for customer learning: get the new feature in quickly with the minimum of effort, see if the users actually use it, and if not then roll back. If the users do like it, then refactor to tidy up any tech debt. Doing all the refactoring first is a complete waste of time if the feature ends up getting rolled back. - Tech debt slows down development but only as a result of delivering early features quicker. It's the sharpening the axe analogy - sooner or later you have to stop chopping the tree to sharpen the axe. Changing the way you chop the tree so you don't have to sharpen the axe is not optimising for getting the most timber quickest. The trick here is communicating with stakeholders about what tech debt is and how it accumulates. They're usually OK with feature pauses if they understand the situation correctly. In other words, like a lot of problems in this industry, this is a communication problem not actually a tech problem.

naasking 2y ago
> Every waterfall project is secretly an iterative project because there's always a Phase 2 where the requirements get updated This is a false equivalence. Waterfall is iterative in requirements but not in code. Changing the requirements document is much, much simpler than changing the code to conform to new requirements. > The optimum amount of tech debt is not zero. Tech debt is a product of changing the product plan/vision/design in reaction to learning from customers. If you're not accruing tech debt then you're not learning from your users. Not sure I agree with the implication "learning from users entails accruing technical debt". Technical debt is a rough measure of the inherent resistance of your program to adapt to customer requirements. Some architectures are flexible enough that you could learn how to configure the architecture differently based on what you've learned from users, but I don't see how that necessarily entails the addition of technical debt, ie. as a rule.

woodpanel 2y ago

Ctrl+F "@TODO"… or have your build pipeline keep track of Todos automatically. Also some IDE integrations keep track and provide quicklinks to lists of TODOs.

yarekt 2y ago
Curious. Got any recommendations for CI tooling that analyses todos? What sort of things that you’ve seen are possible
- shepherdjerred 2y ago
  I read this interesting paper a year or two back about triggerable TODOs in Java: https://users.ece.utexas.edu/~gligoric/papers/NieETAL19TrigIt.pdf https://users.ece.utexas.edu/~gligoric/papers/NieETAL19TrigI...

MeteorMarc 2y ago

The garbage collector already has a 100% coverage test suite.

anymouse123456 2y ago

Wow, what a weird feeling. The authors of this article seem like super smart and experienced guys, but the article itself is introduced with incorrect statements in the very first two sentences and slides downhill from there. "There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all of the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions." 1) There isn't an option to build software by "looking at all of the requirements at once." The requirements of any (sufficiently complex) project will emerge over the course of development, regardless of whether it's built in an iterative style, or with a much larger investment in planning and design up front. We often work iteratively because we acknowledge this particular aspect of reality and it helps everyone when we work closer to reality, rather than fighting it. 2) Technical Debt does not describe design distortions that arise due to "iterative" development, it describes design problems that arise in every project. Quantizing the choices for dealing with tech debt into 4 buckets doesn't feel right either. Changing the design of a running system happens along a continuum and the need and benefit vary widely over the course of any given software lifecycle and surrounding (business or other) environment. Maybe I'm just grumpy this morning, and I think reasonable folks could disagree, but the metaphor at the center of the thesis (tech debt ≅ soon-to-be-deallocated-memory) doesn't hold up for me at all. Tech debt is sometimes left in place, intentionally or not, for many years. Sometimes we chip away at it, sometimes we stop the world and push on it. Sometimes it's bad enough to throw the whole system away and start over. It's like debt for businesses. It can be valuable to take on some debt if it lets you stay alive long enough to pay it back. Finally, I'm struggling with this article because tech debt is already a metaphor, that includes information about how, and when to take it on and pay it down. It's not helpful to layer another, unrelated metaphor on top of it.

gfairbanks 2y ago
Ward Cunningham's original idea of tech debt (see [1], a beauty of concision at just 300 words) is that iterative development distorts your code because you start writing code before you know the requirements, but even so, it's better than waterfall. "The traditional waterfall development cycle has endeavored to avoid programming catastrophy by working out a program in detail before programming begins. We ... [instead use] the alternative, incremental growth ..." Today, the term "tech debt" includes sloppy code, shortcuts, novice code -- really any kind of bad code. The original conception of tech debt is more limited and tied to waterfall vs iterative process choices. [2] [1] Ward Cunningham, The WyCash Portfolio Management System, OOPSLA 1992. https://c2.com/doc/oopsla92.html https://c2.com/doc/oopsla92.html [2] George Fairbanks, Ur-Technical Debt, IEEE Software 2020. https://ieeexplore.ieee.org/document/9121630 https://ieeexplore.ieee.org/document/9121630

igornadj 2y ago

Does anyone else get the feeling that how we deal with technical debt in software is fundamentally and critically misguided? How a pot hole gets filled in isn't a technical challenge, its a political challenge. Developers can talk about the _technical_ side of technical debt all day long, but its a) generally well understood already, and b) blowing in the wind. How technical debt is handled is fundamentally a political problem. We may understand this at a glance, but the solution of "gain knowledge and bring it up at standup" is not only not getting us anywhere, it's deliberately misleading us into thinking its up to us individually to change the organisation. It's as silly as telling the labourer that part of their job is to lobby the city council to spend more time and resources on filling pot holes. Discussions on technical debt should be within the bounds of how to identify organisations where technical debt is identified as a real issue from the top down, and a survival guide for working in organisations where it is not.