7 ms·

The most foolproof way to avoid regressions is to leave the code alone. The best testing is battle testing - if it is working well after plenty of contact with

The most foolproof way to avoid regressions is to leave the code alone. The best testing is battle testing - if it is working well after plenty of contact with the customer then leave the code alone. Bitter experience has taught me this. --- I'm going to expand on my comment a bit - we need to be clearer on what (production) code actually is. We believe it is the definition of the runtime behavior of a given system (and of course it is exactly that). However, for any reasonably complex system, a complete understanding of the definition/behavior is beyond any single person. Thus code is in fact a recording of the social and technical process of producing the system, and as such embeds decisions and solutions that may not be evident to anyone just reading the code. When viewed in this light, it is easy to see how dangerous refactoring can be.

gt565k 3y ago
Sure, but a lot of poorly written spaghetti code that works is sometimes impossible to extend or add new features to without a major refactor or complete rewrite. Currently dealing with some 13 year old objective C code for a mobile app that’s gonna require a major rewrite cause it looks like it was written by someone who should have never been writing code in the first place
- Gibbon1 3y ago
  There is a mathematical technique called simulated annealing. https://en.wikipedia.org/wiki/Simulated_annealing https://en.wikipedia.org/wiki/Simulated_annealing The idea is code goes through development cycles. You have the most leeway to make changes in the first few development cycles. But less and less as time goes on. And the beginning of a development cycle you can make larger changes than at the end. Between cycles the code is frozen and you never touch it unless you can prove a change is needed. And development cycles are triggered by the need to for gross functional changes or interface changes.
  - somewhereoutth 3y ago
    Good analogy. Where I am we try to follow this process for each release cycle - so big changes early, then becoming increasingly conservative as to what gets altered/added as the release date approaches.
dataflow 3y ago
> The most foolproof way to avoid regressions is to leave the code alone. The most foolproof way to avoid car accidents is to avoid ever being around a car. Surely that's not the only objective, right?
- iterateoften 3y ago
  That’s the opposite analogy. The OP is saying drive the car as much as possible and as long as it doesn’t have problems don’t mess around under the hood. Some people tinker with adding mods and things to their cars which increase the probability of the problem.
  - dataflow 3y ago
    > The OP is saying drive the car as much as possible and as long as it doesn’t have problems don’t mess around under the hood. If that's what you and the OP are saying, then you're conflating the codebase with the end product (program). Yes, the end user (at least in an ideal world) shouldn't need to modify the end product. That wasn't ever disputed, though. The point (if I have to use your car-modding analogy instead) is the manufacturer's engineering team very much does need to be able to evolve the design (read: source code) for future versions of the car.
  - koolba 3y ago
    Using the engine analogy, a plane is a better example than a car. Especially as the mental picture of not surviving a crash is more vivid. Would you rather fly on a plane that has been used daily for the same flight for the past 10 years? Or fly on a plane that’s either never flown, just got out of the shop, or hasn’t been flown in years?
    - orwin 3y ago
      If _nothing_ has ever been changed on the plane, I'll take the new one, thank you. I don't know how it lasted that long, but I won't take any chances.
iopq 3y ago
That just leaves the code in a state where it's not clear what's wrong with it, because it works most of the time. There are just two types of code: obviously correct, or not obviously wrong. If you write some code and you don't know how it works, it's most likely not correct in some edge cases. Code that is written to be so simple it's obviously doing the right thing is better than battle tested mess of if statements. I've given interviews of fizzbuzz and even though developers usually recognize they need a separate case for 15, they don't always do the else ifs right (maybe test for the 15 case last when you've already hit the divisible by 3 case first, or similar silly mistakes). In that case, doing the string accumulation refactor makes it clear that the code has no mistakes because it for sure adds a Fizz first and a Buzz second in the 15 case. It's also a good refactor for extending to Bazz (divisible by 7) as well, so it's something you will want to do when you want to add that feature
- MichaelZuo 3y ago
  If that's the typical performance on fizzbuzz, I can't even imagine what the performance would be like on more advanced questions.
  - iopq 3y ago
    Humans are very bad on whiteboard interviews. I like giving people a laptop better now
    - MichaelZuo 3y ago
      The point is to make folks answer without the help of external tools, such as an IDE, on the basis that they will be able to reliable accomplish more advanced tasks with such tools in practice. Whether or not this theory is mostly reliable or barely, or not at all, depends on the viewpoint. If a more advanced question is posed, then the assumption is that the requirement is for the prospective employee to accomplish even more advanced tasks in practice.
      - iopq 3y ago
rizky05 3y ago
it works when the requirements never change. If somebody demands change, you screwed.
- amelius 3y ago
  Or if you suddenly run into a bug that causes the entire thing to collapse for unknown reasons.
marcosdumay 3y ago
You can only ever leave your code alone if it's modular and independent enough so that new changes on your system do not depend on changing it.
sorokod 3y ago
Leaving the code alone, even one that works reasonably well, is not an option. Lehman's laws of software evolution say so. https://en.m.wikipedia.org/wiki/Lehman%27s_laws_of_software_evolution https://en.m.wikipedia.org/wiki/Lehman%27s_laws_of_software_...
zeroonetwothree 3y ago
Refactoring for no reason is not useful. But what happens in practice is you have a new requirement and you realize your hacked together “battle tested” system can’t possibly support it without breaking everything. So then you need to refactor first. And if you are actively developing then refactoring along the way is generally better than if you wait until the very end.
AnimalMuppet 3y ago
True of fixing bugs, too. The odds of introducing a bug when fixing a bug is between 20 and 50%. (Per one study from decades ago. If you can produce better numbers, please do.) But the problem is, even if code is battle-tested, someone always wants it to do something more. To get it to do more means changing it. And when you've changed it, they'll want another change. And then another. So you either don't make changes (even bug fixes!) or you do. If you do, you either refactor the code as you change it, or you don't. If you don't, the code becomes more and more brittle, that is, harder and harder to change without breaking something. So I think you are drawing wrong conclusions from true observations. You are correct that you can break code when you change it, even by refactoring it. Keep it in a state where the probability of that is minimized - where it's clear what the code is doing, and how, and why. To do that, you need to keep refactoring.
joshspankit 3y ago
It’s fantasy, but if each battle was distilled in to a test and added to the test suite then it would be possible to touch production code without breaking business functionality.
- somewhereoutth 3y ago
  Indeed it is a fantasy, however a fantasy that is all too easily and often indulged - which is how regressions end up back with the customer.
RangerScience 3y ago
I mean, yes and no, and aside from that - the article isn't even about regressions. > embeds decisions and solutions that may not be evident to anyone just reading the code. Strongly agree! Good code communicates this to other programmers (including yourself in 3-6 months). If the code doesn't do a good job of this, that alone can not only be worth a refactor, and when you refactor while "what it took to understand what this does" is fresh in your mind, you can write the new version that much more effective at communicating those learnings.
- somewhereoutth 3y ago
  However that requires your learnings to be complete (in respect of the code you are rewriting). My point is that for any reasonably complex code base of some age, that may not actually be possible even with the best of intentions. There will be a corner case you'll miss, or a necessary optimization you'll leave out. Of course this is over a spectrum, so refactoring code that only you wrote quite recently is probably ok and indeed a good thing!
seadan83 3y ago
Testing is a form of battle testing. Code that has been thoroughly tested is akin to a running system for a year. On the other hand, modify that legacy system without test, that "battle test counter" resets to zero and you get to go through the process again. Classically speaking, refactoring is only for code that has thorough coverage. Refactoring code that is not under test might be better worded as rewriting. Obviously, rewriting critical and complex code that has evolved over time with a variety of fixes and random features bolted on with random conditionals - obviously that is risky
- somewhereoutth 3y ago
  > Code that has been thoroughly tested is akin to a running system for a year Except that isn't even remotely true. Perhaps if you system has a purely machine interface, with a limited 'dictionary' of interactions, then a test suite could replicate deployment to a very large degree, but otherwise at best it can give only an indication of how use-worthy the system is. Take a (reasonably complex, with human interface) production system, implement as much testing as you like, and I can guarantee any significant refactoring will introduce a regression, and quite possibly a serious one. The problem with testing is that you need to know what to test. So unless you have some method of capturing every possible way a system is actually used, you will always run into the 'unknown unknowns' that you didn't realize even needed testing.
  - seadan83 3y ago
    > and I can guarantee any significant refactoring will introduce a regression Refactoring suffers from having almost no meaning. I would interpret what you are saying as re-architecture and/or update of system design. That is what most people mean by refactoring today. The article laments that people skip the part of "make it clean" in the "make it work, make it clean, make it fast, in that order" In theory refactoring is a safe activity to do because you are only updating something that has a defined API which is tested. > Except that isn't even remotely true. This is hyperbole. How long before a system is actually fully and truly battle tested? 1 month, 1 year, 5 years? I'm making up numbers; but you could think of it in terms of iterations and code paths. How many executions of a block of logic before you start to trust it more? The passage of time adds more certainty because conditions vary over time. Without any testing, that production release is going to be iteration number one in the journey of battle testing that updated (which can be called re-written) code. All I'm saying is running that code through a battery of tests fast-forwards (some) of that battle testing time. As an analogy, car tires - testing is like running a car tire for a few thousand miles before giving it to the customer. When getting a new one (ie: any update to the code at all), the customer is not getting a new tire with zero miles on it, where perhaps the rubber is bad and that would have been discovered easily after about 10 miles. That car tire might still have issues after 20k miles that are abnormal, but nonetheless the testing released a car tire that had more confidence compared to one with no testing at all. I think my favorite analogy would be more RPG & DnD like games. Testing is like having your resurrected characters start at level 3 instead of level 1. Are they level 9? No, but at the same time it's far better than starting at scratch every time. Every time any modification is done to the code, you don't have to start back at level 1 on your journey to get back to battle tested.
mumblemumble 3y ago
I don't think it's as simple as that. I agree that one should not take something like this lightly. Far, far too many "refactorings" really just boil down to hasty rage-coding at some predecessor who had differing stylistic opinions, or keeners who've never heard of Chesterton's Fence getting frustrated with some annoying design element they don't understand. But also, sometimes the code really is tangled and difficult to maintain, such that it's difficult to make any modifications without introducing regression defects. Concrete example: I recently did a refactoring of a piece of code my team maintained that was so extensive that, by the final iteration, it was arguably a rewrite. The old code was sprawling and complex, could basically only be comprehended by the staff engineers, and tended to be ground zero for defects. The new code is much smaller and simpler, and easier to modify, and every single member of the team is comfortable reading it. - but - I only did this after months of careful observation, quietly building a case for the change and a plan for how to do it. This process included interviewing everyone I thought might know anything about this module, including folks who had moved on to different teams, to make sure I understood as much as possible of what was there and why it was built that way. I discussed my plans with them and sought (and, ultimately always got) their blessing. And then I spent more time bolstering the test coverage, including hunting for edge and corner cases that the existing test suite had missed, when necessary, adding additional tests that did not depend on this module's implementation details so that they would not be broken by the refactor. And then, when I finally did do the actual refactor, I didn't do it all at once. I broke the work into many small, carefully controlled iterations.
- somewhereoutth 3y ago
  Definitely agree! I suppose this is where the 'art' of software engineering comes into play, to know what can and should be refactored, and how to go about it in a safe way.

gt565k 3y ago

Sure, but a lot of poorly written spaghetti code that works is sometimes impossible to extend or add new features to without a major refactor or complete rewrite. Currently dealing with some 13 year old objective C code for a mobile app that’s gonna require a major rewrite cause it looks like it was written by someone who should have never been writing code in the first place

Gibbon1 3y ago
There is a mathematical technique called simulated annealing. https://en.wikipedia.org/wiki/Simulated_annealing https://en.wikipedia.org/wiki/Simulated_annealing The idea is code goes through development cycles. You have the most leeway to make changes in the first few development cycles. But less and less as time goes on. And the beginning of a development cycle you can make larger changes than at the end. Between cycles the code is frozen and you never touch it unless you can prove a change is needed. And development cycles are triggered by the need to for gross functional changes or interface changes.
- somewhereoutth 3y ago
  Good analogy. Where I am we try to follow this process for each release cycle - so big changes early, then becoming increasingly conservative as to what gets altered/added as the release date approaches.

dataflow 3y ago

> The most foolproof way to avoid regressions is to leave the code alone. The most foolproof way to avoid car accidents is to avoid ever being around a car. Surely that's not the only objective, right?

iterateoften 3y ago
That’s the opposite analogy. The OP is saying drive the car as much as possible and as long as it doesn’t have problems don’t mess around under the hood. Some people tinker with adding mods and things to their cars which increase the probability of the problem.
- dataflow 3y ago
  > The OP is saying drive the car as much as possible and as long as it doesn’t have problems don’t mess around under the hood. If that's what you and the OP are saying, then you're conflating the codebase with the end product (program). Yes, the end user (at least in an ideal world) shouldn't need to modify the end product. That wasn't ever disputed, though. The point (if I have to use your car-modding analogy instead) is the manufacturer's engineering team very much does need to be able to evolve the design (read: source code) for future versions of the car.
- koolba 3y ago
  Using the engine analogy, a plane is a better example than a car. Especially as the mental picture of not surviving a crash is more vivid. Would you rather fly on a plane that has been used daily for the same flight for the past 10 years? Or fly on a plane that’s either never flown, just got out of the shop, or hasn’t been flown in years?
  - orwin 3y ago
    If _nothing_ has ever been changed on the plane, I'll take the new one, thank you. I don't know how it lasted that long, but I won't take any chances.

iopq 3y ago

That just leaves the code in a state where it's not clear what's wrong with it, because it works most of the time. There are just two types of code: obviously correct, or not obviously wrong. If you write some code and you don't know how it works, it's most likely not correct in some edge cases. Code that is written to be so simple it's obviously doing the right thing is better than battle tested mess of if statements. I've given interviews of fizzbuzz and even though developers usually recognize they need a separate case for 15, they don't always do the else ifs right (maybe test for the 15 case last when you've already hit the divisible by 3 case first, or similar silly mistakes). In that case, doing the string accumulation refactor makes it clear that the code has no mistakes because it for sure adds a Fizz first and a Buzz second in the 15 case. It's also a good refactor for extending to Bazz (divisible by 7) as well, so it's something you will want to do when you want to add that feature

MichaelZuo 3y ago
If that's the typical performance on fizzbuzz, I can't even imagine what the performance would be like on more advanced questions.
- iopq 3y ago
  Humans are very bad on whiteboard interviews. I like giving people a laptop better now
  - MichaelZuo 3y ago
    The point is to make folks answer without the help of external tools, such as an IDE, on the basis that they will be able to reliable accomplish more advanced tasks with such tools in practice. Whether or not this theory is mostly reliable or barely, or not at all, depends on the viewpoint. If a more advanced question is posed, then the assumption is that the requirement is for the prospective employee to accomplish even more advanced tasks in practice.
    - iopq 3y ago

rizky05 3y ago

it works when the requirements never change. If somebody demands change, you screwed.

amelius 3y ago
Or if you suddenly run into a bug that causes the entire thing to collapse for unknown reasons.

marcosdumay 3y ago

You can only ever leave your code alone if it's modular and independent enough so that new changes on your system do not depend on changing it.

sorokod 3y ago

Leaving the code alone, even one that works reasonably well, is not an option. Lehman's laws of software evolution say so. https://en.m.wikipedia.org/wiki/Lehman%27s_laws_of_software_evolution https://en.m.wikipedia.org/wiki/Lehman%27s_laws_of_software_...

zeroonetwothree 3y ago

Refactoring for no reason is not useful. But what happens in practice is you have a new requirement and you realize your hacked together “battle tested” system can’t possibly support it without breaking everything. So then you need to refactor first. And if you are actively developing then refactoring along the way is generally better than if you wait until the very end.

AnimalMuppet 3y ago

True of fixing bugs, too. The odds of introducing a bug when fixing a bug is between 20 and 50%. (Per one study from decades ago. If you can produce better numbers, please do.) But the problem is, even if code is battle-tested, someone always wants it to do something more. To get it to do more means changing it. And when you've changed it, they'll want another change. And then another. So you either don't make changes (even bug fixes!) or you do. If you do, you either refactor the code as you change it, or you don't. If you don't, the code becomes more and more brittle, that is, harder and harder to change without breaking something. So I think you are drawing wrong conclusions from true observations. You are correct that you can break code when you change it, even by refactoring it. Keep it in a state where the probability of that is minimized - where it's clear what the code is doing, and how, and why. To do that, you need to keep refactoring.

joshspankit 3y ago

It’s fantasy, but if each battle was distilled in to a test and added to the test suite then it would be possible to touch production code without breaking business functionality.

somewhereoutth 3y ago
Indeed it is a fantasy, however a fantasy that is all too easily and often indulged - which is how regressions end up back with the customer.

RangerScience 3y ago

I mean, yes and no, and aside from that - the article isn't even about regressions. > embeds decisions and solutions that may not be evident to anyone just reading the code. Strongly agree! Good code communicates this to other programmers (including yourself in 3-6 months). If the code doesn't do a good job of this, that alone can not only be worth a refactor, and when you refactor while "what it took to understand what this does" is fresh in your mind, you can write the new version that much more effective at communicating those learnings.

somewhereoutth 3y ago
However that requires your learnings to be complete (in respect of the code you are rewriting). My point is that for any reasonably complex code base of some age, that may not actually be possible even with the best of intentions. There will be a corner case you'll miss, or a necessary optimization you'll leave out. Of course this is over a spectrum, so refactoring code that only you wrote quite recently is probably ok and indeed a good thing!

seadan83 3y ago

Testing is a form of battle testing. Code that has been thoroughly tested is akin to a running system for a year. On the other hand, modify that legacy system without test, that "battle test counter" resets to zero and you get to go through the process again. Classically speaking, refactoring is only for code that has thorough coverage. Refactoring code that is not under test might be better worded as rewriting. Obviously, rewriting critical and complex code that has evolved over time with a variety of fixes and random features bolted on with random conditionals - obviously that is risky

somewhereoutth 3y ago
> Code that has been thoroughly tested is akin to a running system for a year Except that isn't even remotely true. Perhaps if you system has a purely machine interface, with a limited 'dictionary' of interactions, then a test suite could replicate deployment to a very large degree, but otherwise at best it can give only an indication of how use-worthy the system is. Take a (reasonably complex, with human interface) production system, implement as much testing as you like, and I can guarantee any significant refactoring will introduce a regression, and quite possibly a serious one. The problem with testing is that you need to know what to test. So unless you have some method of capturing every possible way a system is actually used, you will always run into the 'unknown unknowns' that you didn't realize even needed testing.
- seadan83 3y ago
  > and I can guarantee any significant refactoring will introduce a regression Refactoring suffers from having almost no meaning. I would interpret what you are saying as re-architecture and/or update of system design. That is what most people mean by refactoring today. The article laments that people skip the part of "make it clean" in the "make it work, make it clean, make it fast, in that order" In theory refactoring is a safe activity to do because you are only updating something that has a defined API which is tested. > Except that isn't even remotely true. This is hyperbole. How long before a system is actually fully and truly battle tested? 1 month, 1 year, 5 years? I'm making up numbers; but you could think of it in terms of iterations and code paths. How many executions of a block of logic before you start to trust it more? The passage of time adds more certainty because conditions vary over time. Without any testing, that production release is going to be iteration number one in the journey of battle testing that updated (which can be called re-written) code. All I'm saying is running that code through a battery of tests fast-forwards (some) of that battle testing time. As an analogy, car tires - testing is like running a car tire for a few thousand miles before giving it to the customer. When getting a new one (ie: any update to the code at all), the customer is not getting a new tire with zero miles on it, where perhaps the rubber is bad and that would have been discovered easily after about 10 miles. That car tire might still have issues after 20k miles that are abnormal, but nonetheless the testing released a car tire that had more confidence compared to one with no testing at all. I think my favorite analogy would be more RPG & DnD like games. Testing is like having your resurrected characters start at level 3 instead of level 1. Are they level 9? No, but at the same time it's far better than starting at scratch every time. Every time any modification is done to the code, you don't have to start back at level 1 on your journey to get back to battle tested.

mumblemumble 3y ago

I don't think it's as simple as that. I agree that one should not take something like this lightly. Far, far too many "refactorings" really just boil down to hasty rage-coding at some predecessor who had differing stylistic opinions, or keeners who've never heard of Chesterton's Fence getting frustrated with some annoying design element they don't understand. But also, sometimes the code really is tangled and difficult to maintain, such that it's difficult to make any modifications without introducing regression defects. Concrete example: I recently did a refactoring of a piece of code my team maintained that was so extensive that, by the final iteration, it was arguably a rewrite. The old code was sprawling and complex, could basically only be comprehended by the staff engineers, and tended to be ground zero for defects. The new code is much smaller and simpler, and easier to modify, and every single member of the team is comfortable reading it. - but - I only did this after months of careful observation, quietly building a case for the change and a plan for how to do it. This process included interviewing everyone I thought might know anything about this module, including folks who had moved on to different teams, to make sure I understood as much as possible of what was there and why it was built that way. I discussed my plans with them and sought (and, ultimately always got) their blessing. And then I spent more time bolstering the test coverage, including hunting for edge and corner cases that the existing test suite had missed, when necessary, adding additional tests that did not depend on this module's implementation details so that they would not be broken by the refactor. And then, when I finally did do the actual refactor, I didn't do it all at once. I broke the work into many small, carefully controlled iterations.

somewhereoutth 3y ago
Definitely agree! I suppose this is where the 'art' of software engineering comes into play, to know what can and should be refactored, and how to go about it in a safe way.