10 ms·

> Do not fall into the trap of improving both the maintainability of the code or the platform it runs on at the same time as adding new features or fixing bugs.

by apeace 9y ago

> Do not fall into the trap of improving both the maintainability of the code or the platform it runs on at the same time as adding new features or fixing bugs. I don't disagree at all, but I think the more valuable advice would be to explain how this can be done at a typical company. In my experience, "feature freeze" is unacceptable to the business stakeholders, even if it only has to last for a few weeks. And for larger-sized codebases, it will usually be months. So the problem becomes explaining why you have to do the freeze, and you usually end up "compromising" and allowing only really important, high-priority changes to be made (i.e. all of them). I have found that focusing on bugs and performance is a good way to sell a "freeze". So you want feature X added to system Y? Well, system Y has had 20 bugs in the past 6 months, and logging in to that system takes 10+ seconds. So if we implement feature X we can predict it will be slow and full of bugs. What we should do is spend one month refactoring the parts of the system which will surround feature X, and then we can build the feature. In this way you avoid ever "freezing" anything. Instead you are explicitly elongating project estimates in order to account for refactoring. Refactor the parts around X, implement X. Refactor the parts around Z, implement Z. The only thing the stakeholders notice is that development pace slows down, which you told them would happen and explained the reason for. And frankly, if you can't point to bugs or performance issues, it's likely you don't need to be refactoring in the first place!

jacquesm 9y ago
These problems tend to be systemic, not just tech problems and usually by the time we reach this stage management is a little more amendable to things like feature freezes than what the regular crew would be dealing with. There is a reason you get to that stage. So I can see how we have (much) more freedom when it comes to setting the time table and more diplomacy and better salesmanship might be required at an earlier stage. But then you can point to this comment here and suggest that it is probably much cheaper to do this in house than to hire a bunch of consultants to do it by the time the water is sloshing over the dikes.
- gary__ 9y ago
  Extreme planned refactoring perhaps: "Many teams schedule refactoring as part of their planned work, using a mechanism such as "refactoring stories". Teams use these to fix larger areas on problematic code that need dedicated attention. Planned refactoring is a necessary element of most teams' approach - however it's also a sign that the team hasn't done enough refactoring using the other workflows." https://martinfowler.com/articles/workflowsOfRefactoring/ https://martinfowler.com/articles/workflowsOfRefactoring/
ethbro 9y ago
From personal experience, a good way of approaching the sell to business stakeholders is getting them involved in the bug triage and tracking process. You need to make the invisible (refactoring and code quality) visible (tracking) so they can see what the current state is and map the future. The biggest reason business stakeholders push back against this is that developers tend to communicate this in terms of "You don't need to know anything about this. But we've decided it needs to be done." Which annoys someone when they're paying the hours. I've had decent success with bringing up underlying issues on roadmaps, even to the generalness of "this feature / component has issues." It's a lot easier conversation if it's adding "That thing that we've had on our to-do list for a couple months" vs "This new thing that I never told you about." And as far as pitching, if the code is at all modular, you can usually get away with "new feature in code section A" + "fixes and performance improvements in unrelated section B" in the same release. PS: I love the simple counter-based bookkeeping perspective from the linked post. (And think someone else suggested something similar in a previous performance / debugging front page article)
- jacquesm 9y ago
  > PS: I love the simple counter-based bookkeeping perspective from the linked post. (And think someone else suggested something similar in a previous performance / debugging front page article) If you're short on budget and need to sell the whole package to management just do that one, it will make all that invisible stuff visible in more detail than they likely have the stomach for and you'll be granted budget in no time because there is nothing that spells lost business better than a fair sized gap between customers entering on the left and only a trickle coming out on the right. In effect this is funnel visualization for the internals of an application in all the gory detail.
- crdoconnor 9y ago
  I've tried this "getting them involved" approach and it failed miserably for me. I've tried explaining why module A had to be decoupled from module B to stakeholders. I've tried explaining why we need to set up a CI server. I've tried explaining why technology B needs to isolated and eliminated. In almost all cases they nod and feign interest and understanding and their eyes glaze over. And why should they be interested? The stories are almost always abstract and the ROI is even more abstract. It's all implementation details to them. These stories usually languish near the bottom of the task list and you often need to sneak it in somehow to get it done at all. I think the only real way of dealing with this problem is to allocate time for developers to retrospect on what needs dealing with the code (what problems caused everybody the most pain in the last week?), then time to plan refactoring and tooling stories and time to do those stories alongside features and bugs. Stakeholders do need to assess what level of quality they are happy with (and if it's low, developers should accept that or leave), but that should be limited to telling you how much time to devote to these kinds of stories, not what stories to work on and not what order to do them in. I don't see why they shouldn't have visibility into this process but there's no way they should be allowed to micromanage it any more than they should be dictating your code style guidelines. This is, IMO, the single worst feature of SCRUM - one backlog, 100% determined by the product owner whom you have to plead or lobby if you want to set up a CI server.
crdoconnor 9y ago
>I don't disagree at all, but I think the more valuable advice would be to explain how this can be done at a typical company. I resolved to try this if I ever ran into the same problem again after a whole bunch of arguments at a previous few companies: * Set up a (paper) slider with 0-100% on it and put it somewhere prominent on the wall. Set it at 70%. That's the % of time you spend on features vs. the % of time you spend on refactoring (what that entails should be the development team's prerogative). * Explain to the PM (or their boss) that they can change it at any time. * Explain that it's ok to have it at 100% for a short while but if they keep it up for too long (e.g. weeks) they are asking for a precipitous decline in quality. * Track all the changes and maintain a running average. I think a lot of people suspect that management would just put it at 100% and leave it but I suspect that wouldn't happen. Most manager's "cover their ass" instincts will kick in given how simple, objective and difficult to bullshit the metric is once it's explicit.
- jacquesm 9y ago
  That's a good one, I will definitely steal that. Thank you!
- DougWebb 9y ago
  I'd call it "Maintaining existing code" rather than "Refactoring". To a non-technical manager, the former sounds like pretty much what it is, and won't raise many questions. (If they do question it, ask them if they maintain their car while it's still running ok, or just wait until it breaks down before they do anything to care for it.) Refactoring, on the other hand, sounds like a buzz word, and if they look it up they'll get "rewriting code that's already working so that it continues working the same way". They probably won't get the nuances about why that's a useful thing to do, so it'll sound like busywork and they won't be happy with letting your team do it. They also won't be able to justify it to their management if they're questioned about it, which is critical for getting buy-in from your managers.
bluGill 9y ago
Freeze is a business decision, NOT a technical decision. How much risk is acceptable to them is the question. If they want low risk than they need to freeze early, if they can stand risk then they can freeze latter. If they want the best of both worlds then they need to invest in automation (build and test) up until the point where the costs of automation exceed the value of lower risk with a late freeze date. Remember you need to work in their terms. Risk is something they understand. The risk is they ship as soon as the last feature is done, without discovering that the last feature broke everything else. From there they move back, the last feature is done, so we do a 30 second sanity test - increase that to 30 minutes, 1 week, 1 month... They should have charts (if not create them) showing how long after a bug is introduced is it discovered on average, use those charts to help guide the decision. If the freeze time frame is too long then they allocate budget to fix it, or otherwise plan around this. There are a lot of options, but they are not technical.
dllthomas 9y ago
If a change is hard to make, first make the changes that will make it easy. Recurse.
HolyHaddock 9y ago
> And frankly, if you can't point to bugs or performance issues, it's likely you don't need to be refactoring in the first place! I feel this is a lack of clarity around the word refactoring. Improving the code in a way that fixes bugs is "bug fixing", in a way that makes it do its job faster is "optimisation" and in a way that improves the design is "refactoring". Of course one can do several of them at the same time. And add features, at least in the small. Refactoring can be a valuable activity for bits of a code base where the cost of change could be usefully reduced. It's useful to have a word that can be used to describe that activity that isn't commonly conflated with bug-fixing or optimisation.

by apeace 9y ago

jacquesm 9y ago

These problems tend to be systemic, not just tech problems and usually by the time we reach this stage management is a little more amendable to things like feature freezes than what the regular crew would be dealing with. There is a reason you get to that stage. So I can see how we have (much) more freedom when it comes to setting the time table and more diplomacy and better salesmanship might be required at an earlier stage. But then you can point to this comment here and suggest that it is probably much cheaper to do this in house than to hire a bunch of consultants to do it by the time the water is sloshing over the dikes.

gary__ 9y ago
Extreme planned refactoring perhaps: "Many teams schedule refactoring as part of their planned work, using a mechanism such as "refactoring stories". Teams use these to fix larger areas on problematic code that need dedicated attention. Planned refactoring is a necessary element of most teams' approach - however it's also a sign that the team hasn't done enough refactoring using the other workflows." https://martinfowler.com/articles/workflowsOfRefactoring/ https://martinfowler.com/articles/workflowsOfRefactoring/

ethbro 9y ago

From personal experience, a good way of approaching the sell to business stakeholders is getting them involved in the bug triage and tracking process. You need to make the invisible (refactoring and code quality) visible (tracking) so they can see what the current state is and map the future. The biggest reason business stakeholders push back against this is that developers tend to communicate this in terms of "You don't need to know anything about this. But we've decided it needs to be done." Which annoys someone when they're paying the hours. I've had decent success with bringing up underlying issues on roadmaps, even to the generalness of "this feature / component has issues." It's a lot easier conversation if it's adding "That thing that we've had on our to-do list for a couple months" vs "This new thing that I never told you about." And as far as pitching, if the code is at all modular, you can usually get away with "new feature in code section A" + "fixes and performance improvements in unrelated section B" in the same release. PS: I love the simple counter-based bookkeeping perspective from the linked post. (And think someone else suggested something similar in a previous performance / debugging front page article)

jacquesm 9y ago
> PS: I love the simple counter-based bookkeeping perspective from the linked post. (And think someone else suggested something similar in a previous performance / debugging front page article) If you're short on budget and need to sell the whole package to management just do that one, it will make all that invisible stuff visible in more detail than they likely have the stomach for and you'll be granted budget in no time because there is nothing that spells lost business better than a fair sized gap between customers entering on the left and only a trickle coming out on the right. In effect this is funnel visualization for the internals of an application in all the gory detail.
crdoconnor 9y ago
I've tried this "getting them involved" approach and it failed miserably for me. I've tried explaining why module A had to be decoupled from module B to stakeholders. I've tried explaining why we need to set up a CI server. I've tried explaining why technology B needs to isolated and eliminated. In almost all cases they nod and feign interest and understanding and their eyes glaze over. And why should they be interested? The stories are almost always abstract and the ROI is even more abstract. It's all implementation details to them. These stories usually languish near the bottom of the task list and you often need to sneak it in somehow to get it done at all. I think the only real way of dealing with this problem is to allocate time for developers to retrospect on what needs dealing with the code (what problems caused everybody the most pain in the last week?), then time to plan refactoring and tooling stories and time to do those stories alongside features and bugs. Stakeholders do need to assess what level of quality they are happy with (and if it's low, developers should accept that or leave), but that should be limited to telling you how much time to devote to these kinds of stories, not what stories to work on and not what order to do them in. I don't see why they shouldn't have visibility into this process but there's no way they should be allowed to micromanage it any more than they should be dictating your code style guidelines. This is, IMO, the single worst feature of SCRUM - one backlog, 100% determined by the product owner whom you have to plead or lobby if you want to set up a CI server.

crdoconnor 9y ago

>I don't disagree at all, but I think the more valuable advice would be to explain how this can be done at a typical company. I resolved to try this if I ever ran into the same problem again after a whole bunch of arguments at a previous few companies: * Set up a (paper) slider with 0-100% on it and put it somewhere prominent on the wall. Set it at 70%. That's the % of time you spend on features vs. the % of time you spend on refactoring (what that entails should be the development team's prerogative). * Explain to the PM (or their boss) that they can change it at any time. * Explain that it's ok to have it at 100% for a short while but if they keep it up for too long (e.g. weeks) they are asking for a precipitous decline in quality. * Track all the changes and maintain a running average. I think a lot of people suspect that management would just put it at 100% and leave it but I suspect that wouldn't happen. Most manager's "cover their ass" instincts will kick in given how simple, objective and difficult to bullshit the metric is once it's explicit.

jacquesm 9y ago
That's a good one, I will definitely steal that. Thank you!
DougWebb 9y ago
I'd call it "Maintaining existing code" rather than "Refactoring". To a non-technical manager, the former sounds like pretty much what it is, and won't raise many questions. (If they do question it, ask them if they maintain their car while it's still running ok, or just wait until it breaks down before they do anything to care for it.) Refactoring, on the other hand, sounds like a buzz word, and if they look it up they'll get "rewriting code that's already working so that it continues working the same way". They probably won't get the nuances about why that's a useful thing to do, so it'll sound like busywork and they won't be happy with letting your team do it. They also won't be able to justify it to their management if they're questioned about it, which is critical for getting buy-in from your managers.

bluGill 9y ago

Freeze is a business decision, NOT a technical decision. How much risk is acceptable to them is the question. If they want low risk than they need to freeze early, if they can stand risk then they can freeze latter. If they want the best of both worlds then they need to invest in automation (build and test) up until the point where the costs of automation exceed the value of lower risk with a late freeze date. Remember you need to work in their terms. Risk is something they understand. The risk is they ship as soon as the last feature is done, without discovering that the last feature broke everything else. From there they move back, the last feature is done, so we do a 30 second sanity test - increase that to 30 minutes, 1 week, 1 month... They should have charts (if not create them) showing how long after a bug is introduced is it discovered on average, use those charts to help guide the decision. If the freeze time frame is too long then they allocate budget to fix it, or otherwise plan around this. There are a lot of options, but they are not technical.

dllthomas 9y ago

If a change is hard to make, first make the changes that will make it easy. Recurse.

HolyHaddock 9y ago

> And frankly, if you can't point to bugs or performance issues, it's likely you don't need to be refactoring in the first place! I feel this is a lack of clarity around the word refactoring. Improving the code in a way that fixes bugs is "bug fixing", in a way that makes it do its job faster is "optimisation" and in a way that improves the design is "refactoring". Of course one can do several of them at the same time. And add features, at least in the small. Refactoring can be a valuable activity for bits of a code base where the cost of change could be usefully reduced. It's useful to have a word that can be used to describe that activity that isn't commonly conflated with bug-fixing or optimisation.