7 ms·

Rewriting from scratch is usually, but not universally a bad idea. There are times in your life when you encounter that piece of legacy internal software that

Rewriting from scratch is usually, but not universally a bad idea. There are times in your life when you encounter that piece of legacy internal software that is simply garbage. The math is simple if it takes less time to rewrite than to refactor then rewriting is better, but also keep in mind that estimating software efforts is often inaccurate. I have also noticed that in Agile teams refactoring is generally not rewarded and sometimes is even discouraged. Instead it is a voluntary effort taken at the developer's expense and risk, which is perhaps the greatest crime of Agile.

scarface74 7y ago
Why are you “rewriting” instead of refactoring? Even if the existing code is “garbage”, if you need to add new functionality, you can usually use the “strangler pattern”. Put a nice facade over the old code to provide an interface and write on top of it. If I can do it by putting a COM wrapper on top of 20 year old PowerBuilder code and then interact with it with C#, I can’t imagine a case where this can’t be done.
- Quarrelsome 7y ago
  I was once partially responsible for maintaining an old .NET remoting system that did absolutely nothing to isolate its transactional boundaries. The services were baked into the model which made it easy to write in the first place but very hard to refactor. The code would load a "hollow" entity and then as the entity was interacted with: automagically over the wire the data would be lazy loaded. If you applied an SQL trace you might see over 300 separate SQL commands executed just to load a page that visually represented the entity. This resulted in some dire performance outcomes, a simple for loop over a given collection on the entity could result in N round trips and sometimes even N*N depending on the data modelling in the back end for given types of data within a collection or association. A primary issue of this system is that lots of client code had been written that didn't express its intent. There were about eight relatively complex clients of this software and nowhere in that code base did the client ask for the data it wanted upfront in a way that would allow someone to refactor it without rewriting every single one of those eight clients. Now yes, you could refactor this "architecture-less" sub-system by replacing it with another "architecture-less" sub-system that was "better", but sadly you're left with the same problem: the clients will never tell you what data they need for a use-case they will just assume it will appear when its accessed. To add in those "data demands" you're performing huge shotgun changes across the entire code-base which are always too expensive to justify. One particularly egregious example was a WinForm loading an entity and then stuffing it into the .Tag of a control where a handler would cast it back out later and access it again in a lazy-loading kind of way. Your only hope in that case is some sort of static code analysis to attempt to reverse engineer the client's data demands but that's probably approaching as expensive as any of the changes.
- Bahamut 7y ago
  It's not as easy to just refactor when it comes to UI oftentimes because you are often writing completely new UX.
  - scarface74 7y ago
    True, It was probably bad wording on his part when he said “rewriting from C# to Angular”. If he meant, “we created an API in C# and decoupled the MVC rendering from the business logic and rewrote the front end in Angular”, that’s different. Since he didn’t mention rewriting the back end, I can’t assume they rewrote the backend.
- slowmovintarget 7y ago
  Microservice architectures, if you have the organizational scale to support them, are intended to produce services that can be rewritten and replaced. I've actually done this on purpose, by getting a v0 API into production quickly (1 week), then getting a v1 with all the bells and whistles out with sufficient time. It completely replaces the the v0 microservice which is a separate deployment. The first version provided a specific implementation of the functionality for a single, small user population. The full version scales to millions of users, globally. It is still small enough that we could rewrite it, if we had to. The infrastructure is actually harder to produce (as code) than the API functions themselves. CD pipelines, extensive unit tests, automated smoke tests, automatically configured health monitoring... all of these things make it possible to replace a service with a different implementation that conforms. It's a hard discipline to get to, but once you're there, it is often more economical to rewrite to move on. I guess that's not the case for the usual kinds of projects. We had to spend a year as an organization to get to the point where this was true. It's usually not worth the cost for small teams to do this.
  - Ididntdothis 7y ago
    I always wonder how micro services will do once a refactor doesn’t impact just one service but several services and teams. This seems to be a recipe for a lot of political machinations.
    - scarface74 7y ago
      If you have to change the API signature and not just add more information, you have to version it and have a decent deprecation schedule.
    - slowmovintarget 7y ago
      Part of the expense of microservices is that they be individually replaceable. Every microservice has dependencies to the interface of other services, and only the interface, which is typically RESTful. Refactoring in one code-base (read: one microservice), would never effect another code-base. The unit of reuse in this architecture tends to be the entire API, or libraries published as open source. If there is generic code that is suitable for a library, it is also suitable for contribution. If there is a core business function that should be reused, it is a microservice with an API and gets shared and consumed that way. Our hard rule for refactoring or enhancing services is if it is a breaking change: use another name (Rich Hickey's recommendation). If you provide more or accept less, fine. If you provide less, or require more, use a new route (like a /v2...). Therefore, refactoring in your code base, which makes your life easier, is transparent to everyone else who depends on your code. We have many small teams, so each team is providing 5 to 10 microservices, and refactoring within each independent repo is simple and low impact. There's no magic here, we simply paid the upfront cost of adopting an architecture that can be fielded by a very large dev organization, with a concerted effort in high quality DevOps practices. This move was also a means to scale the organization without dragging our efforts to a screeching halt with maintenance burden.
ayushgp 7y ago
I agree. I recently worked on a price where 1 java class had 14 methods named construct<subtype>fields which all built a html+js+css mess using a string builder. I was asked to add support for 3 more subtypes. And this was the god class for the UI. We rewrote the whole UI in 3 months. There was no way that could be refactored.
- gowld 7y ago
  A rewrite that small is a refactor.
- james_s_tayler 7y ago
  That just sounds like garbage code to begin with.
clktmr 7y ago
> The math is simple if it takes less time to rewrite than to refactor then rewriting is better This is the common pitfall with rewriting. It does not account for the existing software being tested in production for years. There often are subtle behaviors/bugs, that users by now rely on.
- Psyladine 7y ago
  Joel's examples of hairy functions. "Why does this need to be 50 lines and makes half a dozen case exceptions? Just strip out that noise, god who wrote this?" Now legacy systems don't cooperate, well I guess they can just rewrite their APIs too because "software entropy".
  - clarry 7y ago
    So, unmaintainable undocumented untestable legacy code? Yes please, rewrite it, break it, and when its real intended behavior is understood, write tests and document it so the next guy doesn't have to wonder about that crazy function..
    - gowld 7y ago
      Please stay off my systems. If you deploy broken software that runs your business, no one has to wonder about any of the functions any more.
    - blowski 7y ago
      If you keep doing that, there won’t be a next guy. The company will be bankrupt.
      - clarry 7y ago
        That's why commercial software is doomed to be crap.
- nilkn 7y ago
  In some cases, it's possible to rewrite incrementally via refactoring. (Whether you can do this depends on the architecture of the system and whether the starting and target languages have decent interoperability.) In other words, rewrite one small piece of a system at a time while maintaining the entire unit and integration test suite. While this doesn't completely obviate the risks you point out, being able to rely on existing tests can make the effort far more plausible and less risky.
BurningFrog 7y ago
> I have also noticed that in Agile teams refactoring is generally not rewarded and sometimes is even discouraged As a long time Agilist, the spread of Fake Agile is a sad thing to see. Constant refactoring is a core Agile practice! https://martinfowler.com/bliki/OpportunisticRefactoring.html https://martinfowler.com/bliki/OpportunisticRefactoring.html
- jcadam 7y ago
  Every "Agile" shop I've worked in was awful and has been doing it wrong according to Agile fans. "That wasn't true Agile" is starting to sound about as credible as "That wasn't true Communism." Best methodology: Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project. That requires two things: 1. Hiring the right people. 2. Trusting them. Most companies aren't willing to do this, so... we get a semi-unstructured form of micro-management hell which goes by the name "Agile." Regardless of whatever that term originally meant, this is what it means now.
  - Ididntdothis 7y ago
    “Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project.” That’s what “Agile” is when you read the Agile Manifesto. It seems every reasonable methodology quickly gets perverted by leadership to a top down, dogmatic dictatorship caricature of itself. Happened to OOP, Agile, will probably happen to FP. You could argue that what Lenin and friends did in Russia was also not what Marx had in mind.
    - thu2111 7y ago
      This is what agile proponents always say. Whatever someone is suggesting, that's agile. It appears to mean nothing. Here, you're saying that the agile manifesto literally says nothing beyond "let people make up whatever process they want", which boils down to nothing at all. If that's really the case then agile as a concept may as well not exist. But we have story points, epics, scrums, retros etc. Agile is very much more than that.

scarface74 7y ago

Why are you “rewriting” instead of refactoring? Even if the existing code is “garbage”, if you need to add new functionality, you can usually use the “strangler pattern”. Put a nice facade over the old code to provide an interface and write on top of it. If I can do it by putting a COM wrapper on top of 20 year old PowerBuilder code and then interact with it with C#, I can’t imagine a case where this can’t be done.

Quarrelsome 7y ago
I was once partially responsible for maintaining an old .NET remoting system that did absolutely nothing to isolate its transactional boundaries. The services were baked into the model which made it easy to write in the first place but very hard to refactor. The code would load a "hollow" entity and then as the entity was interacted with: automagically over the wire the data would be lazy loaded. If you applied an SQL trace you might see over 300 separate SQL commands executed just to load a page that visually represented the entity. This resulted in some dire performance outcomes, a simple for loop over a given collection on the entity could result in N round trips and sometimes even N*N depending on the data modelling in the back end for given types of data within a collection or association. A primary issue of this system is that lots of client code had been written that didn't express its intent. There were about eight relatively complex clients of this software and nowhere in that code base did the client ask for the data it wanted upfront in a way that would allow someone to refactor it without rewriting every single one of those eight clients. Now yes, you could refactor this "architecture-less" sub-system by replacing it with another "architecture-less" sub-system that was "better", but sadly you're left with the same problem: the clients will never tell you what data they need for a use-case they will just assume it will appear when its accessed. To add in those "data demands" you're performing huge shotgun changes across the entire code-base which are always too expensive to justify. One particularly egregious example was a WinForm loading an entity and then stuffing it into the .Tag of a control where a handler would cast it back out later and access it again in a lazy-loading kind of way. Your only hope in that case is some sort of static code analysis to attempt to reverse engineer the client's data demands but that's probably approaching as expensive as any of the changes.
Bahamut 7y ago
It's not as easy to just refactor when it comes to UI oftentimes because you are often writing completely new UX.
- scarface74 7y ago
  True, It was probably bad wording on his part when he said “rewriting from C# to Angular”. If he meant, “we created an API in C# and decoupled the MVC rendering from the business logic and rewrote the front end in Angular”, that’s different. Since he didn’t mention rewriting the back end, I can’t assume they rewrote the backend.
slowmovintarget 7y ago
Microservice architectures, if you have the organizational scale to support them, are intended to produce services that can be rewritten and replaced. I've actually done this on purpose, by getting a v0 API into production quickly (1 week), then getting a v1 with all the bells and whistles out with sufficient time. It completely replaces the the v0 microservice which is a separate deployment. The first version provided a specific implementation of the functionality for a single, small user population. The full version scales to millions of users, globally. It is still small enough that we could rewrite it, if we had to. The infrastructure is actually harder to produce (as code) than the API functions themselves. CD pipelines, extensive unit tests, automated smoke tests, automatically configured health monitoring... all of these things make it possible to replace a service with a different implementation that conforms. It's a hard discipline to get to, but once you're there, it is often more economical to rewrite to move on. I guess that's not the case for the usual kinds of projects. We had to spend a year as an organization to get to the point where this was true. It's usually not worth the cost for small teams to do this.
- Ididntdothis 7y ago
  I always wonder how micro services will do once a refactor doesn’t impact just one service but several services and teams. This seems to be a recipe for a lot of political machinations.
  - scarface74 7y ago
    If you have to change the API signature and not just add more information, you have to version it and have a decent deprecation schedule.
  - slowmovintarget 7y ago
    Part of the expense of microservices is that they be individually replaceable. Every microservice has dependencies to the interface of other services, and only the interface, which is typically RESTful. Refactoring in one code-base (read: one microservice), would never effect another code-base. The unit of reuse in this architecture tends to be the entire API, or libraries published as open source. If there is generic code that is suitable for a library, it is also suitable for contribution. If there is a core business function that should be reused, it is a microservice with an API and gets shared and consumed that way. Our hard rule for refactoring or enhancing services is if it is a breaking change: use another name (Rich Hickey's recommendation). If you provide more or accept less, fine. If you provide less, or require more, use a new route (like a /v2...). Therefore, refactoring in your code base, which makes your life easier, is transparent to everyone else who depends on your code. We have many small teams, so each team is providing 5 to 10 microservices, and refactoring within each independent repo is simple and low impact. There's no magic here, we simply paid the upfront cost of adopting an architecture that can be fielded by a very large dev organization, with a concerted effort in high quality DevOps practices. This move was also a means to scale the organization without dragging our efforts to a screeching halt with maintenance burden.

ayushgp 7y ago

I agree. I recently worked on a price where 1 java class had 14 methods named construct<subtype>fields which all built a html+js+css mess using a string builder. I was asked to add support for 3 more subtypes. And this was the god class for the UI. We rewrote the whole UI in 3 months. There was no way that could be refactored.

gowld 7y ago
A rewrite that small is a refactor.
james_s_tayler 7y ago
That just sounds like garbage code to begin with.

clktmr 7y ago

> The math is simple if it takes less time to rewrite than to refactor then rewriting is better This is the common pitfall with rewriting. It does not account for the existing software being tested in production for years. There often are subtle behaviors/bugs, that users by now rely on.

Psyladine 7y ago
Joel's examples of hairy functions. "Why does this need to be 50 lines and makes half a dozen case exceptions? Just strip out that noise, god who wrote this?" Now legacy systems don't cooperate, well I guess they can just rewrite their APIs too because "software entropy".
- clarry 7y ago
  So, unmaintainable undocumented untestable legacy code? Yes please, rewrite it, break it, and when its real intended behavior is understood, write tests and document it so the next guy doesn't have to wonder about that crazy function..
  - gowld 7y ago
    Please stay off my systems. If you deploy broken software that runs your business, no one has to wonder about any of the functions any more.
  - blowski 7y ago
    If you keep doing that, there won’t be a next guy. The company will be bankrupt.
    - clarry 7y ago
      That's why commercial software is doomed to be crap.
nilkn 7y ago
In some cases, it's possible to rewrite incrementally via refactoring. (Whether you can do this depends on the architecture of the system and whether the starting and target languages have decent interoperability.) In other words, rewrite one small piece of a system at a time while maintaining the entire unit and integration test suite. While this doesn't completely obviate the risks you point out, being able to rely on existing tests can make the effort far more plausible and less risky.

BurningFrog 7y ago

> I have also noticed that in Agile teams refactoring is generally not rewarded and sometimes is even discouraged As a long time Agilist, the spread of Fake Agile is a sad thing to see. Constant refactoring is a core Agile practice! https://martinfowler.com/bliki/OpportunisticRefactoring.html https://martinfowler.com/bliki/OpportunisticRefactoring.html

jcadam 7y ago
Every "Agile" shop I've worked in was awful and has been doing it wrong according to Agile fans. "That wasn't true Agile" is starting to sound about as credible as "That wasn't true Communism." Best methodology: Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project. That requires two things: 1. Hiring the right people. 2. Trusting them. Most companies aren't willing to do this, so... we get a semi-unstructured form of micro-management hell which goes by the name "Agile." Regardless of whatever that term originally meant, this is what it means now.
- Ididntdothis 7y ago
  “Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project.” That’s what “Agile” is when you read the Agile Manifesto. It seems every reasonable methodology quickly gets perverted by leadership to a top down, dogmatic dictatorship caricature of itself. Happened to OOP, Agile, will probably happen to FP. You could argue that what Lenin and friends did in Russia was also not what Marx had in mind.
  - thu2111 7y ago
    This is what agile proponents always say. Whatever someone is suggesting, that's agile. It appears to mean nothing. Here, you're saying that the agile manifesto literally says nothing beyond "let people make up whatever process they want", which boils down to nothing at all. If that's really the case then agile as a concept may as well not exist. But we have story points, epics, scrums, retros etc. Agile is very much more than that.