6 ms·

"Rewriting a system from the ground up is essentially an admission of failure as a designer. It is making the statement, “We failed to design a maintainable sys

"Rewriting a system from the ground up is essentially an admission of failure as a designer. It is making the statement, “We failed to design a maintainable system and so must start over.” " Or maybe the domain you are working with was vastly more complex than you imagined, so your nice little system turned out to be more of a prototype than a finished product. Or maybe the features needed was vastly different from what you expected when you started implementing the thing. I'm sure everyone can come up with a million other reasons. But the problem I have every time with these "Never rewrite your software" stories. Is that they are almost always one-sided. You rarely hear about the stories where a company decided not to do a rewrite and was out-innovated because every little feature took months to implement. Or failed because they could not attract competent developers because their codebase was a complete mess. Yes, the rewrite might have been a partial or complete failure. But would the patient have survived without the operation anyway?

nradov 7y ago
Or maybe your customer base and system load has expanded by an order of magnitude beyond the original design.
- save_ferris 7y ago
  Writing scalable software is all about being as flexible as possible. If system load expanded by an order of magnitude and the original design wasn't flexible enough to scale with the load, that's a bad software design.
  - wildrhythms 7y ago
    Sure but then risk being dinged by the customer/management for "over-engineering something simple." How do you determine where the cut off point is?
  - TuringNYC 7y ago
    Or perhaps the first project was budgeted to only handle limited scale and prove feasibility. In which case it was a success, and now that feasibility is proven, scale can be budgeted for. I mean, if people tried to build for mega scale from day one, every pie in the sky idea I have would start with Spark clusters and load balancers and Kubernetes...which obviously is all expensive, time consuming, and may simply not be worth the cost given the business objective.
  - paulddraper 7y ago
    !! Designing SQLitec data store and designing Cassandra data store are rather different.
    - pessimizer 7y ago
      True, but initially decoupling the business logic from the specifics of data retrieval would allow you to write an abstraction layer when you need to switch stores rather than having to rewrite the entire thing.
      - nomel 7y ago
        Fighting for the independence of these separate pieces and layers of abstraction is one of the most difficult problems I’ve had being a developer. Management always wants time saving, “well if we just make these parts dependent, the fix will be much quicker”, requiring deep explanations that requires justifying some “hypothetical” scenario in the future (which always ends up being in a few months). Other developers, not familiar with the deeper understanding of the problem being solved, knowingly or unknowingly implement the shortcuts, adding technical debt and hidden complexity. I don’t know what the balance is, but I find the only way to make a long lived system is to fight hard for the separation of concerns, or slowly fall prey to chaos.
        AnimalMuppet 7y ago
        > (which always ends up being in a few months) I have seen something about technical debt that says that it's a win over the span of weeks, but a loss by the time of months.
  - firethief 7y ago
    Worrying about scalability before you need it is a good way to ensure that you won't.
  - twblalock 7y ago
    If you try to build the most flexible, scalable solution on day one, you are very likely to make the wrong predictions about the architecture you will need in the future. You will probably need to rewrite the software anyway.
  - crimsonalucard 7y ago
    A lot of people might disagree, and this is just my opinion, but if you are using object oriented programming than in 99% of all cases you are not being as flexible as possible. Tying methods to data is an inflexibility. With sufficient complexity under OOP you will always be writing code that feels like it is poorly designed because the paradigm promotes poor design and lack of flexibility.
    - kaoD 7y ago
      > Tying methods to data is an inflexibility. Tying methods to data is an inevitability (whether it is on run- or compile- time). Can you give me an example of a function that does not depend on the data it operates on? What makes OOP wrong is not tying methods to data, but tying data to ontologies. Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence.
      - nradov 7y ago
        You can accomplish exactly that in OOP with careful use of multiple inheritance or interfaces.
        kaoD 7y ago
        That's kinda my point. Interfaces should be the main abstraction. I don't think multiple inheritance (or inheritance at all) is particularly useful. Composition over inheritance. Interfaces over inheritance. I don't care what something "is" and I don't think ontologies (classes, inheritance) are particularly useful as a means of abstraction.
      - crimsonalucard 7y ago
        >Tying methods to data is an inevitability (whether it is on run- or compile- time). Can you give me an example of a function that does not depend on the data it operates on? I mean tying methods to data via a class. A instantiated class is a structure that has data and has methods specific to operating on said data. Either way, any function that takes void input and with any return type is an example of a function that does not depend on data, but that wasn't my point. Note that a function is different from a method contextually speaking. >What makes OOP wrong is not tying methods to data, but tying data to ontologies. Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence. We may be talking about the same thing. Though I'm not sure what you mean by this: "Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence." Can you please clarify?
- goto11 7y ago
  This should not require a ground-up rewrite though, unless the code is a big ball of mud in the first place. You might need to change the overall architecture (e.g split monolithic code into to multiple services) and/or optimize some critical bottlenecks, but most code (business logic, UI etc.) will be unaffected.
michaelvoz 7y ago
Rewriting is a very healthy part of software engineering. Sometimes you just didn’t anticipate what the company or product was going to do. The solution you wrote for 1k/qps probably won’t work at a million qps, and it is kind of silly to expect it to.
bdcravens 7y ago
Sometimes when you're designing a system, you acknowledge it's bad (ie, technical debt). Ultimately, your purpose is supporting the system (typically the business), not crafting artisanal design and code. I suspect there are many beautiful git repos that are deleted when companies close their doors.
randallsquared 7y ago
> I'm sure everyone can come up with a million other reasons. Indeed. One reason for rewrite is that there isn't a smooth continuum of changes from vCurrent to vFuture. If the discontinuity will have to be bridged anyway, and if other intermediate changes aren't going to bring sufficient improvement, jumping into it might be the most efficient way.
- gfodor 7y ago
  In finance there is the concept of "Risk Adjusted Returns" -- in the case of a re-write, time efficiency is not a sufficient conceptual framework for talking about it, because it's risk laden. For example, I would much rather take 2x as long to accomplish the goal, if its 10x less likely to result in catastrophic failure. A failure to recognize the risk vs time tradeoff I think can lead to a failure of being motivated enough to "work the problem" -- often there is a continuum of changes that exists -- and it might not be "smooth" and could be in fact quite painful -- but it will be way, way less risky. It's often so less risky to transition software incrementally vs bet the farm on a rewrite, I feel like in most contexts a rewrite should just be ruled out completely without further discussion: better to just muddle through and try to figure out a way to survive without it, since you may come up with a better solution that is less risk laden just by waiting (due to new ideas, new technology, new market opportunities, etc) and not stalling your current product development and risking that you'd effectively destroy everything of value you've created so far due to total stagnation.
  - convivialdingo 7y ago
    On my 5th startup - I cannot agree more. You only have so much cashflow - your first priority is to maintain and deliver for your customers. They pay the bills usually. Architecture and engineering are important, and they need support and time to rework problems. But incremental improvement is almost always the best choice. If you have the cash then you should consider not only what’s best for product maintenance, but also how you could use that cash to grow your business.
crimsonalucard 7y ago
I would say 90% of the time, there is a failure in design. Most of the time the designer does not fully understand the problem because the product hasn't been tested yet. Additionally even after the product has been rewritten 90% of the time the new design is wrong. Often the second design is slightly better or worse than the first design because the second design is also a shot in the dark. Why? This is because there is no way to theoretically prove a design is the correct solution to the problem. The only way to prove a design is to test it. There is no science or proof behind "design" only "design principles" and anecdotes... all of which can be broken and tend to evolve over time. This is why designers tend to be artists rather than scientists. Designers are hired for problem spaces where no theoretical solution exist. If most designs are wrong how do products even exist? Well computers are fast enough to cover up design flaws. Hacks can be made to bridge gaps in designs. Overall designs are wrong but can be patched to make anything work.
yeahitslikethat 7y ago
Or, did a rewrite and it launched them to success.
hinkley 7y ago
That doesn't sound like an 'or' to me. That sounds like a 'why'. A rewrite also means no new features for a very long time. And my very cynical view is that people asking for a rewrite are motivated by the peace and quiet that happens during the first six months of the rewrite. In a world where people leave after 3 years and get a pass for their first few months that represents a pretty large fraction of their tenure. But it's a bit like rehab. You've decided to start over with a new set of habits and hope it sticks. But like rehab or new year's resolutions, the relapse rate is high. My current take is that there is a paradox here. The teams that deserve a rewrite usually don't need a rewrite. There's a high degree of overlap between being capable of fixing your broken crap properly and being able to refactor to a better system. The only rewrites I've seen work look like the Ship of Theseus. As far as management is concerned it's not a new system. But the old developers can barely recognize the code.

nradov 7y ago

Or maybe your customer base and system load has expanded by an order of magnitude beyond the original design.

save_ferris 7y ago
Writing scalable software is all about being as flexible as possible. If system load expanded by an order of magnitude and the original design wasn't flexible enough to scale with the load, that's a bad software design.
- wildrhythms 7y ago
  Sure but then risk being dinged by the customer/management for "over-engineering something simple." How do you determine where the cut off point is?
- TuringNYC 7y ago
  Or perhaps the first project was budgeted to only handle limited scale and prove feasibility. In which case it was a success, and now that feasibility is proven, scale can be budgeted for. I mean, if people tried to build for mega scale from day one, every pie in the sky idea I have would start with Spark clusters and load balancers and Kubernetes...which obviously is all expensive, time consuming, and may simply not be worth the cost given the business objective.
- paulddraper 7y ago
  !! Designing SQLitec data store and designing Cassandra data store are rather different.
  - pessimizer 7y ago
    True, but initially decoupling the business logic from the specifics of data retrieval would allow you to write an abstraction layer when you need to switch stores rather than having to rewrite the entire thing.
    - nomel 7y ago
      Fighting for the independence of these separate pieces and layers of abstraction is one of the most difficult problems I’ve had being a developer. Management always wants time saving, “well if we just make these parts dependent, the fix will be much quicker”, requiring deep explanations that requires justifying some “hypothetical” scenario in the future (which always ends up being in a few months). Other developers, not familiar with the deeper understanding of the problem being solved, knowingly or unknowingly implement the shortcuts, adding technical debt and hidden complexity. I don’t know what the balance is, but I find the only way to make a long lived system is to fight hard for the separation of concerns, or slowly fall prey to chaos.
      - AnimalMuppet 7y ago
        > (which always ends up being in a few months) I have seen something about technical debt that says that it's a win over the span of weeks, but a loss by the time of months.
- firethief 7y ago
  Worrying about scalability before you need it is a good way to ensure that you won't.
- twblalock 7y ago
  If you try to build the most flexible, scalable solution on day one, you are very likely to make the wrong predictions about the architecture you will need in the future. You will probably need to rewrite the software anyway.
- crimsonalucard 7y ago
  A lot of people might disagree, and this is just my opinion, but if you are using object oriented programming than in 99% of all cases you are not being as flexible as possible. Tying methods to data is an inflexibility. With sufficient complexity under OOP you will always be writing code that feels like it is poorly designed because the paradigm promotes poor design and lack of flexibility.
  - kaoD 7y ago
    > Tying methods to data is an inflexibility. Tying methods to data is an inevitability (whether it is on run- or compile- time). Can you give me an example of a function that does not depend on the data it operates on? What makes OOP wrong is not tying methods to data, but tying data to ontologies. Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence.
    - nradov 7y ago
      You can accomplish exactly that in OOP with careful use of multiple inheritance or interfaces.
      - kaoD 7y ago
        That's kinda my point. Interfaces should be the main abstraction. I don't think multiple inheritance (or inheritance at all) is particularly useful. Composition over inheritance. Interfaces over inheritance. I don't care what something "is" and I don't think ontologies (classes, inheritance) are particularly useful as a means of abstraction.
    - crimsonalucard 7y ago
      >Tying methods to data is an inevitability (whether it is on run- or compile- time). Can you give me an example of a function that does not depend on the data it operates on? I mean tying methods to data via a class. A instantiated class is a structure that has data and has methods specific to operating on said data. Either way, any function that takes void input and with any return type is an example of a function that does not depend on data, but that wasn't my point. Note that a function is different from a method contextually speaking. >What makes OOP wrong is not tying methods to data, but tying data to ontologies. Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence. We may be talking about the same thing. Though I'm not sure what you mean by this: "Limiting a function to operating on what something "is" is an artificial limitation. Limiting a function to operating on what something "has" or "can do" is a logical consequence." Can you please clarify?
goto11 7y ago
This should not require a ground-up rewrite though, unless the code is a big ball of mud in the first place. You might need to change the overall architecture (e.g split monolithic code into to multiple services) and/or optimize some critical bottlenecks, but most code (business logic, UI etc.) will be unaffected.

michaelvoz 7y ago

Rewriting is a very healthy part of software engineering. Sometimes you just didn’t anticipate what the company or product was going to do. The solution you wrote for 1k/qps probably won’t work at a million qps, and it is kind of silly to expect it to.

bdcravens 7y ago

Sometimes when you're designing a system, you acknowledge it's bad (ie, technical debt). Ultimately, your purpose is supporting the system (typically the business), not crafting artisanal design and code. I suspect there are many beautiful git repos that are deleted when companies close their doors.

randallsquared 7y ago

> I'm sure everyone can come up with a million other reasons. Indeed. One reason for rewrite is that there isn't a smooth continuum of changes from vCurrent to vFuture. If the discontinuity will have to be bridged anyway, and if other intermediate changes aren't going to bring sufficient improvement, jumping into it might be the most efficient way.

gfodor 7y ago
In finance there is the concept of "Risk Adjusted Returns" -- in the case of a re-write, time efficiency is not a sufficient conceptual framework for talking about it, because it's risk laden. For example, I would much rather take 2x as long to accomplish the goal, if its 10x less likely to result in catastrophic failure. A failure to recognize the risk vs time tradeoff I think can lead to a failure of being motivated enough to "work the problem" -- often there is a continuum of changes that exists -- and it might not be "smooth" and could be in fact quite painful -- but it will be way, way less risky. It's often so less risky to transition software incrementally vs bet the farm on a rewrite, I feel like in most contexts a rewrite should just be ruled out completely without further discussion: better to just muddle through and try to figure out a way to survive without it, since you may come up with a better solution that is less risk laden just by waiting (due to new ideas, new technology, new market opportunities, etc) and not stalling your current product development and risking that you'd effectively destroy everything of value you've created so far due to total stagnation.
- convivialdingo 7y ago
  On my 5th startup - I cannot agree more. You only have so much cashflow - your first priority is to maintain and deliver for your customers. They pay the bills usually. Architecture and engineering are important, and they need support and time to rework problems. But incremental improvement is almost always the best choice. If you have the cash then you should consider not only what’s best for product maintenance, but also how you could use that cash to grow your business.

crimsonalucard 7y ago

I would say 90% of the time, there is a failure in design. Most of the time the designer does not fully understand the problem because the product hasn't been tested yet. Additionally even after the product has been rewritten 90% of the time the new design is wrong. Often the second design is slightly better or worse than the first design because the second design is also a shot in the dark. Why? This is because there is no way to theoretically prove a design is the correct solution to the problem. The only way to prove a design is to test it. There is no science or proof behind "design" only "design principles" and anecdotes... all of which can be broken and tend to evolve over time. This is why designers tend to be artists rather than scientists. Designers are hired for problem spaces where no theoretical solution exist. If most designs are wrong how do products even exist? Well computers are fast enough to cover up design flaws. Hacks can be made to bridge gaps in designs. Overall designs are wrong but can be patched to make anything work.

yeahitslikethat 7y ago

Or, did a rewrite and it launched them to success.

hinkley 7y ago

That doesn't sound like an 'or' to me. That sounds like a 'why'. A rewrite also means no new features for a very long time. And my very cynical view is that people asking for a rewrite are motivated by the peace and quiet that happens during the first six months of the rewrite. In a world where people leave after 3 years and get a pass for their first few months that represents a pretty large fraction of their tenure. But it's a bit like rehab. You've decided to start over with a new set of habits and hope it sticks. But like rehab or new year's resolutions, the relapse rate is high. My current take is that there is a paradox here. The teams that deserve a rewrite usually don't need a rewrite. There's a high degree of overlap between being capable of fixing your broken crap properly and being able to refactor to a better system. The only rewrites I've seen work look like the Ship of Theseus. As far as management is concerned it's not a new system. But the old developers can barely recognize the code.