13 ms·

Refactor vs. Rewrite

198 points by ntietz 6y ago

megameter 6y ago
IME whether you rewrite or refactor, the lesson is the same: You have to grind your way into the good architecture. It doesn't become good because the code is fresh, but because you have battle scars to show. And a success story in that case comes from having a complete learning loop. In a lot of orgs the learning itself is argued against for one reason or another - development proceeds with as little feedback on quality as can be gotten away with.
- jeffbee 6y ago
  I feel like you have denied the existence of people who are capable of writing down a good architecture on the first try, without all the grinding.
  - sdenton4 6y ago
    No good plan survives first contact with the enemy... Less pithily, the good architecture will almost certainly end up with arbitrary patches and madness after 2 years of active use, if anyone cares about it at all... And then somebody will join and say 'This is a mess, we should refactor and/or burn it to the ground.'
    - jeffbee 6y ago
      I don't doubt that this could accurately reflect your experience in the industry, but it is not a universal truth. I've worked on sustaining engineering of systems that were years old, the stewardship of which involved refactoring only, and which are still in production today. There are lots of systems that are more or less right in version 1.
      - MaulingMonkey 6y ago
        > There are lots of systems that are more or less right in version 1. While true, in my experience this usually happened when at least one - if not all three - of these conditions were true: 1. The problem domain is relatively static, unmoving, and self contained 2. The systems in question is relatively small 3. The developers in question have written substantially similar systems before, or ground locally, or used version 0.x version numbers first (in any of these cases, is it really "version 1" per se?) ...game development is admittedly weak in all 3 categories, and high on time pressure. Oh sure, there's some solid code that will probably last decades in many codebases, but even the best programmers will occasionally make a system that - while good in isolation - needs reworking when combined with other systems - to say nothing of the content and rendering pipelines that vary wildly by decade.
        jjeaff 6y ago
        4. A system that never needs to scale much beyond its original projected load.
        recursive 6y ago
        First read as "original project lead", but I think that could apply too.
        ratww 6y ago
        That's certainly true in my experience. I've seen some great, readable, codebases written by solo developers that had lots of previous experience writing code in larger teams. Caveat: it's never something that's too large or complex, of course.
        toast0 6y ago
        0. The requirements were known in advance of starting the design.
      - WalterBright 6y ago
        > the stewardship of which involved refactoring only I've known several such systems, and the reason they stuck with the original architecture is they wrote it in C, and it's very hard to iterate architecture in C programs. The programmers will just keep bashing the C code so it works well enough. Yes, I know this is a provocative statement. The reason behind it is that C's ability to encapsulate behavior is just not very good. Even simple things, like "is it an array or a linked list" leak out all over the place. Even simpler, "is it a value or a reference type" means one has to swap "." and "->" everywhere. C++ half-fixed it by introducing reference types, D fixes it all the way by noting that "." can be unambiguously used for both operations.
      - Ididntdothis 6y ago
        To some degree that's matter of luck. Sometimes all your assumptions are correct from the start and the requirements don't change much. But often a lot of things change over time until the first architecture doesn't fit anymore. I would agree there are better and worse architects but luck definitely plays a role.
      - goto11 6y ago
        Yeah it is always important in such discussions to remember how vastly different development can be. Developing embedded software for an appliance have vastly different constraints compared to a web app startup which pivot every three months.
    - constantine42 6y ago
      It's funny to say, but I can't agree. I've designed and implemented many custom systems for which there was never a predecessor, and not only did it work splendidly at first release, but have been extended and enhanced many times over the years with no refactoring. Many of these systems have been extended enough to start taking over other systems that were never intended. Design a system to be modular from the very beginning with only the most fundamental of API signatures and you win. Many of these signatures may even no-op for years, but because they're fundamental to the problem, you know there could be a case where they're needed. I've never had a failed or late project and most of my projects are ones that were assigned to me because no other team would touch them due to all of the attention and required SLAs.
    - twhitmore 6y ago
      Nonsense. Competent designers with 8 or 10 years experience should absolutely be capable of architecting a solid system that is cleanly extensible for forseeable business requirements. Probably one huge confounding factor, however, is the continual drive by juniors (or supposed seniors who think like juniors) to use the latest & shiniest technologies. These of course these are unknown to the team, of questionable long-term suitability and introduced without knowledge of the "right" way to use them. In the article the OP describes using Elixir without A) needing it or B) being able to handle its downsides. Plus many random versions of Angular and React. Plus the overheads of an excessive focus on SOA. My emphasis in architecture is on using simple & strong tools, and delivering great architecture in the problem space. Configurability, extensibility and DSLs are my forte. I don't need a new language -- I know how to use the ones I've got.
  - WalterBright 6y ago
    I have yet to know any such people :-)
    - inopinatus 6y ago
      Agreed. Every new design is informed by past mistakes. Programming is not an a priori field.
    - Ididntdothis 6y ago
      Just avoid touching anything complex :-)
      - moksly 6y ago
        Or a deadline.
  - xwdv 6y ago
    Certainly possible to make a good one on the first try. A great one? Probably not, unless you are doing something absolutely trivial and done several times before, like a CRUD app.
  - BurningFrog 6y ago
    I'd suspect anyone who claims that is a bit full of himself. Maybe their work isn't so great to other people. Or maybe it is. I won't rule it out. But I also think that anyone who does this probably works too hard. It's so much easier to build things incrementally.
    - collyw 6y ago
      >It's so much easier to build things incrementally. I would say its much easier if you know what you are building in advance. Building incrementally is usually more about exploring ambiguous ideas and not having a clear vision in my experience.
      - BurningFrog 6y ago
        I'm always building something I've never built before. And the requirements always change as you go. So knowing what I'm building in advance never really happens.
  - deleted 6y ago
    [deleted]
  - Sammi 6y ago
    "Everyone has a plan, until they get hit" - Mike Tyson
  - deleted 6y ago
    [deleted]
- KKKKkkkk1 6y ago
  Here's a nice quote from Jony Ive to support your point Much of the design process is a conversation, a back-and-forth as we walk around the tables and play with the models. [Steve Jobs] doesn't like to read complex drawings. He wants to see and feel a model. He's right. I get surprised when we make a model and then realize it's rubbish, even though based on CAD renderings it looked great. He loves coming in here because it's calm and gentle. It's a paradise if you're a visual person. There are no formal design reviews, so there are no huge decision points. Instead, we can make the decisions fluid. Since we iterate every day and never have dumb-ass presentations, we don't run into major disagreements.
  - nikofeyn 6y ago
    i'm not sure ive has a lot of lessons to give on evolving an architecture in a way that improves maintainability and adds functionality. ive's entire MO revolved around removing those two things from the systems he designed.
- arkh 6y ago
  Have you heard of our lord and savior, Domain Driven Design? Here are the blue and red books.
  - r4mbini 6y ago
    Are these books Evans(blue) and Vernon(red)?
    - arkh 6y ago
      Yes. I think Vernon's is the best as it has more emphasis on the process and organisation instead of the code.
      - r4mbini 6y ago
        Interesting. I’ve only read the blue one, will get a copy of red. Thank you!
- logicchains 6y ago
  >IME whether you rewrite or refactor, the lesson is the same: You have to grind your way into the good architecture. I think that only applies to things nobody on the team's built before. From my domain, HFT, if we hire a senior dev who's built blazing-fast futures arbitrage systems working for other companies, chances are if he rewrote our system it'd have a very nice architecture from the get-go, because he's already solved a similar problem multiple times before (and learned from the mistakes in those architectures).
  - Someone 6y ago
    And then, that senior dev will pick a newer, better CPU, on-device bus, network topology, or your setup simply has a different scale, and hard-learned heuristic “don’t use more than x foo’s for y” or “the network is the bottleneck; use huge machines” become obsolete, and your dev will get some new battle scars. I guess that will happen in HFT all the time, as you don’t want that blazing-fast setup, you want something even faster.
- constantine42 6y ago
  "Grind" your way into good architecture? Good architecture is designing, not trial-and-errored. The only grinding I do is mental before I type a single letter of code.
  - TheSpiceIsLife 6y ago
    So all the code you’ve ever written was perfect in every way from day one? You’ve never made a suboptimal choice? And never made a mistake?
readme 6y ago
Refactor is a good word to use if you don't want to upset the coworker who's code you completely replaced.
3pt14159 6y ago
I rewrote a codebase for a major stock exchange in about four months. It was originally in Ruby, which is my most proficient language, and my implementation was in Python, my second most proficient language. The original Ruby codebase had been worked on for years and was a complete mess. Not because of Ruby, but because the people that developed it were a little sloppy or junior or time pressured. Who knows? It's totally possible to do a rewrite and to do it well. The advantage of having a pre-existing implementation is that it's far, far easier to write tests before you implement because most edge-cases are covered in the existing implementation. Make sure you do a feature freeze before you start, then once you're ready for production do both shadow testing and canary testing and things will go well.
- majormajor 6y ago
  > The advantage of having a pre-existing implementation is that it's far, far easier to write tests before you implement because most edge-cases are covered in the existing implementation. Make sure you do a feature freeze before you start, then once you're ready for production do both shadow testing and canary testing and things will go well. There are two ways I've seen it go vastly over schedule when these things bite you. 1) There might be edge cases covered in the original implementation that neither you nor current stakeholders remember or know about. So discovery can either take way longer than expected (until you're confident you know 100% of what the code does) or implementation can get constantly delayed by "oh, yeah, we do still need that." 2) Feature freezes are a rare gift. Usually it's "we need this new thing" which leads to engineering saying "that would be hard to do in the current system, we should rewrite it" so now you have to rebuild all the old stuff plus build the new thing. And its requirements will probably keep changing on you.
- deleted 6y ago
  [deleted]
- celim307 6y ago
  God bless whoever wrote tests before you. Whether it be a rewrite or working on the current code base, those who have the time and resources and don't write tests are inexcusable and bad stewards
sys_64738 6y ago
When a developer says "I am going to rewrite ..." you have to question why? If you were not doing this "rewrite" then what else would you be doing? Refactor is usually a term used to minimally modify legacy code to allow new functionality to be added. Rewriting implies rewriting legacy code (regressions) and wasting time. If a developer says "it's the only way" then you need to be sure you can trust their judgement.
- esperent 6y ago
  I frequently write code that is publically available on GitHub and used by other people. Sometimes my first attempt at structuring the code works out well (more and more often as I grow in experience). Sometimes... Not so much. Usually a refactor is enough, but occasionally I have to buckle down and say, to be ready for public consumption, a rewrite would be faster.
- to11mtm 6y ago
  I tend to agree. Part of the problem is when developers get thrown into unfamiliar code bases but can't read code well, they may think about just doing a rewrite. But really, Thoughtful refactoring is better. And as another comment noted, in many cases good architecture comes from learning by past mistakes of yourself and others. OTOH, there are edge cases. I'm currently finishing up a rewrite, but it was due to the existing software being written in a language that most people aren't proficient in and isn't really the norm in our company. I've witnessed but did not participate in two rewrites. In one case, a company wanted to switch languages and databases (got sick of oracle licensing.) In another case, the company had a pretty good justification for a rewrite; the system was a monstrosity of .NET on top of VB6. In fact, in this case after the initial rewrite attempt failed and there was a change in management we pivoted strategy. Instead of trying to rewrite the world we would just continue to refactor things as we made changes to those modules, and only hard-rewrite the VB6 code. I don't know if they're still doing that, but it was a -lot- more productive than the rewrite team spending a year producing nothing that ever made it to production.
- ydnaclementine 6y ago
  > In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."
brilee 6y ago
This blog post doesn't seem to mention how long the process took - that's what I'm most interested in!
- jt2190 6y ago
  Yes, this is the key warning of Spolsky’s post [1]: The rewrite team had invested years of work and still had nothing shippable. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
tjr 6y ago
My team inherited a 70%-complete AngularJS application, the user interface for a passengers and crew on a new series of airplanes. It worked fine enough on modern tablets and phones, but there were also some interface devices hardwired into the planes, which would be very challenging to update at this point, and on those devices the software ran very slow. Through several experiments, we determined that the sluggishness was unfortunately coming from AngularJS. We tried some performance improvements, but it wasn't getting very far. We built some prototypes using other Javascript frameworks, and found VueJS gave us the best performance. Management was concerned about us rewriting. The end customer was concerned about us rewriting. So risky! But we saw no other way to deliver adequate software performance. We got done on time, with great system performance, to the accolades of a very happy end customer.
- momokoko 6y ago
  I’m my career, UI rewrites have been By far the most successful. It may just be anecdotal, but my assumption has been that most of the edge cases are more easily identified because UIs are by nature going to have most of their business logic visible to the human eye.
k__ 6y ago
A good rewrite looks like a refactor. Strangle that monolith and none will be the wiser.
- ratww 6y ago
  I love that answer. Strangling for me is the best of both worlds: you can do it iteratively/cautiously but at the same time you can start "fresh" if necessary/possible.
- teddyh 6y ago
  Conversely, a good refactoring process eventually results in a complete effective rewrite.
WalterBright 6y ago
I constantly refactor my code, even the ancient stuff. Here's my most recent one: https://github.com/dlang/dmd/pull/11202 https://github.com/dlang/dmd/pull/11202 which changes a data structure from a linked list to an array.
- mattkerle 6y ago
  I was about to comment something sarcastic, then I googled you and instead I'll just say thanks!
  - WalterBright 6y ago
    I don't mind if you say something sarcastic, but I am happy to say "you're welcome!"
pan69 6y ago
Maybe controversial but I think original authors should be able to rewrite, i.e. they can learn from previous mistakes and incorporate those lessons into the rewrite.
- Cthulhu_ 6y ago
  Oh yeah they definitely should, after all they will also have a lot of domain knowledge which new developers have to relearn (I'm in that situation myself at the moment). But, they need to have actually learned since then. They need to prove that if given another shot, they can do better. I've been able to do that, we built an application in BackboneJS, then Angular became big; we used that in a separate application, then decided to gradually rewrite the original in Angular (we were able to ship it to customers during the rewrite, rewriting things on a per-page basis without doing much with the design). It took a lot longer than the original, but also because new features were added and we were doing things much neater, with proper unit tests and the like. We learned that we had underestimated the complexity of a single page application, that it's not just a matter of "show whatever the back-end is returning so make sure the back-end is tested, front-end doesn't really matter much".
mamcx 6y ago
Refactoring/Rewrites is the MOST rewarding task that can be achieved with both legacy and new code. I do it from the start of my career (when I don't even know the name of it). And that include, regulary, rewriting to different langs (like, for example FoxPro -> C#, C# -> Python -> F# -> Swift (aborted) -> Rust (noob) -> Rust (now I know rust, almost!) <-- yes, the same project, years on it. Exits a lot of FUD against this. Is risky? Is hard? Can be messed up? Everything is staked against you? Yep LIKE ANY OTHER CODE ACTIVITY. ---- Things that help in make this a success (assuming, of course, an average, half-decent developer. But I have done it when I not even know anything, so who knows if it work?): CRITICAL: - Have source control - Have a task manager (mine: pivotal) - Split stuff in small tasks (couple hours max) - Priorize to know the data(schemas), the business REAL priorities (because all the time, everything will be declared of imperative importance). The code? Less so. If the code is good then you are not considering this at all. - This can be done in the small or in the large. So, a full rewrite can be totally done in hours. If all is so tangled this is not true, then the code is beyond salvation, REWRITE. Nice: - If possible, deployable code in one go (of the old code) - The database (or data in general) is kind of easy to grasp (if not, all is a mess, REWRITE). With a good database, rewrite is far more profitable than refactoring messy logic. - Have good test that can be used to validate it? Great. I rarely have that luxury. To become good: - Rewrite/Refactor is a muscle. The more you do it, the better you become. - Do A LOT of MICRO side-projects or experiments. Trash them, rewrite them, explore them. Make your mind used to it (note: this is not "ruin my life, only code yo!", is to start a major idea with a micro side project before commit bigly) Everyone know the second time your code is better. Imagine if you do 5 rewrite in successions. That code will be celestial! - Be confident in being able to read code. Read code. And read more. Much more if is a new lang. - Ask question, and ask the same question, differently. - Pick a project that you can do in your sleep (mine: Pseudo micro ORMS) and start with that when new lang or framework. In a side project. Must be of small size (few files, at most) - If change lang, framework, be aware that some are BETTER than others. Learn to know why and when start using it. - Dedicate a few minutes learning useful stuff. - Do the HARDEST stuff first, at least, a significant piece of it. If the hard is done, the rest is easier! - Have a solid selection of companion tools that you RARELY need to change (mine: Sqlite, Postgres, nginx, ubuntu (deploy)) The MORE messed up, the BIGGER is the payoff: - Not have tests or are all useless? Start writing test and all is tangled? Rewrite - Like a stupid idiot you are 4 months into a refactoring (doing stuff properly and all) and all paths lead to doom? Rewrite (you probably feel it 3 months ago. Idiot me) - You testing (manually, auto, whatever) take X times than write code? Rewrite - The customer is on fire, nothing work, all things are delayed? Rewrite - Everyone in the path of requirements are even more lost than you? Rewrite --- Surely exist some stuff that is conveniently ignored in this list: Have a customer/team that accept the rewrite. Look, if the customers/boss/team is THAT bad, then NOT MATTER what you do, you ARE SCREWD. But I'm lucky, and have worked in situations with serious trouble (but decent humans around, even if can't help much and I must figure a lot on my own) have tell me that people are more accepting of rewrites than some could think. Even delayed projects can be tolerated if exist a good flow of improvements. P.D: This is my experience in my sector, and the few I have done in a mid-size startup...
satyrnein 6y ago
So they got off of Elixir and microservices, but their introduction of Typescript and GraphQL didn't pay off like they hoped, and now they need to get off GraphQL. I think the lesson of "Choose boring technology" applies here. One of the reasons rewrites are risky is that very often the new tech doesn't live up to expectations.
- bsaul 6y ago
  yeap, this is a good point. This is the first time i see a rewrite to "older". Older architecture, older language, older methodology. And the only part that serm to not have met expectation are the ones using new stuffs ( graphql and maybe typescript). Maybe there's a rule here : complete rewrite is ok if you're rewriting to "older" tech.
- devonkim 6y ago
  If the business model is boring, you’ll need complex technology to compensate. If your business model is complex like in a lot of enterprise software, you should strive to keep the technology as simple as possible. Simple doesn’t mean to not use complex or cutting edge things IMO but to be conservative in using things that push the cognitive load of your collective team. A 10 person company probably doesn’t need a K8S cluster for a mobile app but I’d be surprised if there isn’t one for a 10k+ software company.
  - jspash 6y ago
    "If the business model is boring, you’ll need complex technology to compensate." Could you elaborate on this? What is the complexity compensating for? Boredom of the developers?
    - ansgri 6y ago
      I suppose the boring business model needs some secret sauce, i.e. advanced technology, to stay competitive -- since it's boring, many people are likely to do it, the margins are thin, you need to optimize.
    - 6y ago
bdcravens 6y ago
I feel there's room for something in-between (reimplementation? resurrection?): start with a clean slate, and bringing in large swaths of functionality and code and then rebuilding other parts. I think this is useful when you have a lot of work to upgrade an older version of your core framework or many dependencies, you're changing a large piece of your infrastructure (like a database), or you have a lot of work to "fix" your test suite.
cryptonector 6y ago
Some rewrites can be easy because a better technology exists that makes it easy. For example, you might have a NoSQL DB with lots of hand-rolled queries, and then you get a query language for it, so now you can rewrite a great deal of code. You still hand-roll transactions because eventual consistency, CAP, and all that, but then you get a SQL DB that... gives you all of that anyways but with good ole SQL (with eventual consistency, not ACID, but hey, CAP), so now you can rewrite the queries and the transactions, and maybe that's the lion's share of the app, so you've essentially rewritten the app. What about the REST layer? Well, you get inspired by PostgREST and write a similar tool for your new DB and there's nothing left of the original.
Ericson2314 6y ago
I think everyone is missing the big point here: they current implementation was in an untyped language and so refactoring was just to hard. Use languages with good type systems and then refactoring is easy, and always the correct technical choice. Let me say that again, the killer app of type systems is that no mistake can "total" the code.
- Chris_Newton 6y ago
  More expressive type systems might make refactoring safer, but they don’t necessarily make it easier. The former happens because it’s harder to change something encoded in the types accidentally. For the latter, it should be easier to change something encoded in the types deliberately, but often the opposite is true.
  - jillesvangurp 6y ago
    Having a type system enables zero risk automated refactorings, even complex ones and make them a no brainer. Not having that builds a reluctance to do even simple refactorings. A good example is the simplest possible refactoring: renaming things. I was doing this in pycharm on a simple python project the other day and it proposed modifying just about all dependencies on the classpath because it couldn't tell apart things that were in scope and out of scope of the refactoring. I've seen similar things happen on javascript and ruby codebases. Renaming things is a PITA in those languages. Not safe at all. On any Kotlin or Java code base I do this all the time without thinking twice. I rename stuff, I move stuff, extract variables, auto fix things, etc. It just happens. A rename is a complete non issue for that. Doesn't matter if it's a local variable or the package name of your entire code base. You can trivially modify thousands of lines of code with a keystroke without breaking stuff.
    - Chris_Newton 6y ago
      It seems to me that what you’re talking about there has more to do with having clear rules in the language for scope and modularity than to do with the type system.
nottsdev 6y ago
I think rewriting is a great option, provided both options are weighed up appropriately. However saying this, we made the decision to rewrite a legacy system around a year ago and the project got parked due to "urgent" projects being fed to our team. This was frustrating but we could not account for this external factor when weighing up the options.
- gonzus 6y ago
  I think changing priorities is a risk that has to be taken into account whenever you are considering any task that will extend for a long time -- rewriting code is just one example. If you have to go on for many weeks / months on a project without the possibility of showing intermediate results, it is naive not to think that there is a high risk of the project being cancelled.
  - nottsdev 6y ago
    I completely agree with you, it is just a complicated metric to factor in. I guess one of the frustrations was the fact that our team had been been assigned with assurance that we will not be affected by changing priorities.
gampleman 6y ago
A really undervalues idea in software development is architecting software for rewriting it in the future. If you make your software small and well defined, then it becomes a reasonable investment to simply rewrite it from scratch. This is a great talk about this idea: https://vimeo.com/108441214 https://vimeo.com/108441214
- Wildgoose 6y ago
  Optimise for Deletion! I always say this, but it's difficult to get people to understand just how important a concept this is.
jillesvangurp 6y ago
It's always a judgment call. But in general, there are a few things that make the choice easy. Any UI project, I tend to look at the expected shelf life of the thing knowing full well that most UI projects get discarded pretty quickly. Stuff like backbone, angular 1, etc. were pretty popular a few years ago and you don't hear a lot about people whining about refactoring those. Mostly those code bases have already been changed; several times probably or abandoned. So, if we have a bit of UI and it's important for it to be right and it is clearly not and I have a team of developers leaning towards unceremoniously dumping the old code base, I'm not going to object very long or hard. The reality is that it probably can be fixed but all the people interested in doing that have gone. For backend systems it's different. I regularly see code bases with long histories. Often a good sign of how good they are is if people can work on them with confidence. In those cases, I lean towards doing more incremental changes. Update dependencies, modernize a few things even if they've been fine, etc. If that's not the case, it's a net cost sink and something needs to be done.
franciscop 6y ago
> a handful of engineers focused on machine learning (ML). [...] We leaned into a singular moderator talking to a group of people. I am curious about why an early chat/conference company needs so many ML Engineers? What are you using them for? (the Remesh mentioned in the article seems to be https://remesh.chat/ https://remesh.chat/ and not https://remesh.ai/ https://remesh.ai/, based on previous articles, the logo and blog domain)
albertzeyer 6y ago
I think Google even has a policy that almost all parts of code in their mono-repo needs to be rewritten (not just refactored) every 2 years or sth like that.
- satyrnein 6y ago
  I guess nobody felt like rewriting Google Reader!
  - WJW 6y ago
    Not "significantly technically challenging" enough, so no promotion opportunities there.
cientifico 6y ago
In my experience, refactor vs rewrite is the wrong approach. Knowledge of the business must be maintained. If knowledge is on the team because the company is stable, rewrite. 37 signals is a clear example of this. If the knowledge is in the code, because multiple people or teams worked on the code base over the past few years, refactor. A rewrite will be the recipe for disaster. I never found an exception to this rule.
- dagss 6y ago
  I have been responsible for an exception to this rule. We were new people coming in to take over, decided to go for rewrite, and were successful. But part of the reason for the rewrite was that the logic (translation of business domain knowledge into code) in the old version was wrong resulting in unstable behavior. It could have been fixed in the old version (very many places) but the lack of a test suite too tipped the scales for rewrite (with test suite). You can say that we learned the domain (at least) as well as the authors of original system after coming in, but won't that be the case of any successful rewrite?..
  - cientifico 6y ago
    If you are able to get the domain knowledge fast enough, sure. Or as you said, the domain knowledge has changed from the original.
  - rakoo 6y ago
    It seems to me it's still not contradictory to GP: domain knowledge wasn't in the code, it was outside. The thing didn't work in all cases so you had to get the specs from something other than the code; rewriting according to this exterior source was the sensible thing to do
- chosenbreed37 6y ago
  Great heuristic!
Cthulhu_ 6y ago
I'm in the middle of a rewrite, but because my employer also wants me to do work on the existing application, I end up doing a lot of refactoring as well - if only for my own sanity and to help me understand the existing code. We're talking an only 8 year old (continuously developed during that time) app built in PHP and JS, but by someone who never bothered to learn web development proper. Resulting in a lot of copy / pasted code, string concatenation to produce HTML, etc. The problem isn't so much that, it's that the original developer was also highly productive, so the codebase exploded to nearly 200K LOC. They recognized the existing UI was not going to work, so after the previous developer left, they put out a job advert to which I was recruited for. The job being to rewrite the application with modern technologies, with stuff like structure and tests built in this time. I was okay with that, I mean I could probably do some things with the existing UI but it would be very labor intensive and if it was ever finished, it would still be a mid-2000's-looking Dojo app. I've (for now) decided on Go and React, deploying it as a self-contained application (our distribution is via RPM packages on bare metal or virtual servers). Not sure about Go at the moment, most work is straight data manipulation (think REST API, database and writing XML files), struggling a bit with things like data consistency (because Go won't enforce you to set all fields, mind you neither does Java but you can have Lombok generate constructors and the like). Anyway, my problem right now is that I was hired under the premise that maintenance of the old app would be like a day a week at most. Of course, in practice they're asking me to add new features to the existing application, so I've been spending over half my time on it instead, not making much headway with the rebuild. My fear right now is that they'll scrap the new UI in favor of just keeping the old one around. But there was an article a while ago that pointed things like that out, I believe it was titled something like "why rewrites fail". Very close to home. I mean I still believe a rewrite will be beneficial, but it's going to take at least another year and a half I think before it's feasible to replace the existing UI. Should have a chat with the boss whether we can hire more people to work on it and / or the existing application.
- satyrnein 6y ago
  If it hits the same database, build the new features in the new stack, and launch them. Users should click from the old to the new without even realizing. You have to figure out some details like sharing sessions, but this way avoids adding onto the old application. Otherwise you'll never catch up.
barking 6y ago
I don't know if this is off-topic or not. More times than I care to admit I've lost hours of coding due to crashing without saving first. Without exception when I redo the lost code it's far better designed than the original and usually (not always) in a lot less time. I'd almost be inclined to recommend always throwing away your first completed effort and rewriting it while your mind's fresh.
eddieoz 6y ago
Probably it should consider the alignment to the business strategy. Developers tend to rewrite because of tech challenges. But I agree with the conclusion and rewrite should be considered if it is impacting the strategy.
battery423 6y ago
I would argue that rewrite is slightly more difficult then refactor. Refactor means, after all, making the current system better while rewrite would mean no update/change/feature on the current system and more pressure to get it done. I would herby say that a good team with experienced people will succeed in both cases while an less experienced team will probably have a higher chance of doing a refactoring then a rewrite. I like refactorings better. It allows me to optimize a system slow and steady. I like rewriting better if you don't need to support the old env while doing the rewrite and having expierence in the old system.
fabian2k 6y ago
I found the move away from GraphQL again interesting, maybe because it supports some of my concerns about it. I never looked in detail into GraphQL, but just from the general principles I understand I found it rather scary to decide to use it. It's a big complicated thing directly between the data and the frontend. If anything goes wrong there, now I have to debug a much more complex piece of software than if I use a REST API. And just like in the article, based on my superficial knowledge I was concerned that it could be easy to create bad queries, and hard to fix performance issues. I'm not saying GraphQL is bad, I really don't know enough about it. But given the central position it takes, I would not feel comfortable deciding to use it without a serious amount of prototyping and getting to understand how it works. It often feels weird to me that such a complex piece in this kind of critical position is often casually recommended for any kind of web application.
- emerongi 6y ago
  > it could be easy to create bad queries, and hard to fix performance issues You can do complexity analysis of the query before you run it and deny requests that have complexity above a certain value. Some graphQL libraries provide this out of the box. Of course it won't eliminate all possibilities of bad queries, but at least most of them. For me GraphQL simplifies life quite a bit. We have an internal library that plugs a lot of things in automatically but leaves full customizability options, which makes it quite convenient to build APIs. On the front-end there are very useful libraries that again make life simpler. In short: the biggest benefit is that I have more time to spend on things that actually matter, instead of inventing my own protocol that in the end is just going to be a worse graphQL.
dirtylowprofile 6y ago
My last 3 clients wanted me to rewrite their React Native apps back to native iOS and Android like in a separate codebase. I’m not sure if this is a trend but their common complaint was that they are sick of the performance of the app and each platfrom has a unique bug. Also this one project written in RN took like a year to develop.
blackpanda 6y ago
I think we need to focus on even more basic point while deciding between refactor vs. rewrite. That is whether the basic DB schemas/Data Structures are well designed/suited to current business understanding or not. Because rest of the structure stands on top of them. I do not see how you can refactor stuff if the majority of the DB schema/Data Structures are built incorrectly and/or not suited to current understanding/layout of the business. You will have to rewrite. The only case where refactor can possibly work is that basic DB schema/Data Structures is 90% good and changes are only required at the business logic level.
code-faster 6y ago
Rewriting has a concave payoff. It requires high levels of certainty to be valuable. The bigger the rewrite, the more likely you're less certain than you think. Refactoring is the same as rewriting, just on a smaller scale, which increases certainty. Joel's advice assumes you will overestimate your certainty.
dep_b 6y ago
I hear more often things like "oh I don't have experience with X, so let's not use it". I mean programming languages are easy, frameworks are hard. Like learning how to use Django (which is pretty easy as far as frameworks go) to me seems harder than using React (assuming I know that already) with TypeScript instead of JavaScript. I would probably be really happy to use TypeScript instead. If Elixir was a really bad match because it was hard to use the required libraries, OK I can understand that.
- commandlinefan 6y ago
  > programming languages are easy, frameworks are hard That's been my observation as well, but I find myself surrounded by folks who insist that any framework should be a ten-minute tutorial away from mastery (even people who ought to know better).
aazaa 6y ago
> ... We had very poor test coverage in the areas most in need of refactoring because they were the oldest code, written before we established good test practices. That's probably not a coincidence. I like to ask those most strongly opposed to test-first development how they refactor code. I have yet to get an answer the leads to anything other than a rewrite at some point. Of course, the rewrite might be coming for the test-first codebase as well. But in that case, the test suite can be leveraged to great effect as a detailed specifications document. This is one of the benefits of test suites that doesn't get enough attention.
- bearjaws 6y ago
  How is this a test-first argument? You can write tests at anytime, bad discipline is bad discipline. I've never worked somewhere that didn't write tests because "they didn't write them first", they didn't write them because they were an immature organization. My current job leaves it up to the developers, and most don't write tests first. Its no miracle that despite us not writing tests first, all changes have their unit tests, integration tests, and end to end tests. This is due to the fact its baked into our SDLC and enforced in code review.
  - commandlinefan 6y ago
    > You can write tests at anytime Actually, every single Java enterprise application I've worked on for the past 20 years has been test-resistant: the developers declare everything "public static" so none of the application can be run without running all of the rest of it (since each class imports another that transitively imports everything else, all with static initializers that read configuration files and connect to live database instances). Every refactor/rewrite I've ever suggested has been specifically to allow tests to be written.
- ratww 6y ago
  Wait, this is confusing. By test-first do you mean TDD? Even if you don't do TDD you gotta have tests before refactoring, I've never seen anyone writing tests after refactoring. I don't think this is controversial at all. Anyone please correct me if I'm wrong. (I've seen refactoring without testing but that's another matter)
namelessoracle 6y ago
AngularJS seems like the number one culprit to cause people to raise their hands and go with the rewrite over refactor approach.

198 points by ntietz 6y ago

IME whether you rewrite or refactor, the lesson is the same: You have to grind your way into the good architecture. It doesn't become good because the code is fresh, but because you have battle scars to show. And a success story in that case comes from having a complete learning loop. In a lot of orgs the learning itself is argued against for one reason or another - development proceeds with as little feedback on quality as can be gotten away with.

jeffbee 6y ago
I feel like you have denied the existence of people who are capable of writing down a good architecture on the first try, without all the grinding.
- sdenton4 6y ago
  No good plan survives first contact with the enemy... Less pithily, the good architecture will almost certainly end up with arbitrary patches and madness after 2 years of active use, if anyone cares about it at all... And then somebody will join and say 'This is a mess, we should refactor and/or burn it to the ground.'
  - jeffbee 6y ago
    I don't doubt that this could accurately reflect your experience in the industry, but it is not a universal truth. I've worked on sustaining engineering of systems that were years old, the stewardship of which involved refactoring only, and which are still in production today. There are lots of systems that are more or less right in version 1.
    - MaulingMonkey 6y ago
      > There are lots of systems that are more or less right in version 1. While true, in my experience this usually happened when at least one - if not all three - of these conditions were true: 1. The problem domain is relatively static, unmoving, and self contained 2. The systems in question is relatively small 3. The developers in question have written substantially similar systems before, or ground locally, or used version 0.x version numbers first (in any of these cases, is it really "version 1" per se?) ...game development is admittedly weak in all 3 categories, and high on time pressure. Oh sure, there's some solid code that will probably last decades in many codebases, but even the best programmers will occasionally make a system that - while good in isolation - needs reworking when combined with other systems - to say nothing of the content and rendering pipelines that vary wildly by decade.
      - jjeaff 6y ago
        4. A system that never needs to scale much beyond its original projected load.
        recursive 6y ago
        First read as "original project lead", but I think that could apply too.
        ratww 6y ago
        That's certainly true in my experience. I've seen some great, readable, codebases written by solo developers that had lots of previous experience writing code in larger teams. Caveat: it's never something that's too large or complex, of course.
      - toast0 6y ago
        0. The requirements were known in advance of starting the design.
    - WalterBright 6y ago
      > the stewardship of which involved refactoring only I've known several such systems, and the reason they stuck with the original architecture is they wrote it in C, and it's very hard to iterate architecture in C programs. The programmers will just keep bashing the C code so it works well enough. Yes, I know this is a provocative statement. The reason behind it is that C's ability to encapsulate behavior is just not very good. Even simple things, like "is it an array or a linked list" leak out all over the place. Even simpler, "is it a value or a reference type" means one has to swap "." and "->" everywhere. C++ half-fixed it by introducing reference types, D fixes it all the way by noting that "." can be unambiguously used for both operations.
    - Ididntdothis 6y ago
      To some degree that's matter of luck. Sometimes all your assumptions are correct from the start and the requirements don't change much. But often a lot of things change over time until the first architecture doesn't fit anymore. I would agree there are better and worse architects but luck definitely plays a role.
    - goto11 6y ago
      Yeah it is always important in such discussions to remember how vastly different development can be. Developing embedded software for an appliance have vastly different constraints compared to a web app startup which pivot every three months.
  - constantine42 6y ago
    It's funny to say, but I can't agree. I've designed and implemented many custom systems for which there was never a predecessor, and not only did it work splendidly at first release, but have been extended and enhanced many times over the years with no refactoring. Many of these systems have been extended enough to start taking over other systems that were never intended. Design a system to be modular from the very beginning with only the most fundamental of API signatures and you win. Many of these signatures may even no-op for years, but because they're fundamental to the problem, you know there could be a case where they're needed. I've never had a failed or late project and most of my projects are ones that were assigned to me because no other team would touch them due to all of the attention and required SLAs.
  - twhitmore 6y ago
    Nonsense. Competent designers with 8 or 10 years experience should absolutely be capable of architecting a solid system that is cleanly extensible for forseeable business requirements. Probably one huge confounding factor, however, is the continual drive by juniors (or supposed seniors who think like juniors) to use the latest & shiniest technologies. These of course these are unknown to the team, of questionable long-term suitability and introduced without knowledge of the "right" way to use them. In the article the OP describes using Elixir without A) needing it or B) being able to handle its downsides. Plus many random versions of Angular and React. Plus the overheads of an excessive focus on SOA. My emphasis in architecture is on using simple & strong tools, and delivering great architecture in the problem space. Configurability, extensibility and DSLs are my forte. I don't need a new language -- I know how to use the ones I've got.
- WalterBright 6y ago
  I have yet to know any such people :-)
  - inopinatus 6y ago
    Agreed. Every new design is informed by past mistakes. Programming is not an a priori field.
  - Ididntdothis 6y ago
    Just avoid touching anything complex :-)
    - moksly 6y ago
      Or a deadline.
- xwdv 6y ago
  Certainly possible to make a good one on the first try. A great one? Probably not, unless you are doing something absolutely trivial and done several times before, like a CRUD app.
- BurningFrog 6y ago
  I'd suspect anyone who claims that is a bit full of himself. Maybe their work isn't so great to other people. Or maybe it is. I won't rule it out. But I also think that anyone who does this probably works too hard. It's so much easier to build things incrementally.
  - collyw 6y ago
    >It's so much easier to build things incrementally. I would say its much easier if you know what you are building in advance. Building incrementally is usually more about exploring ambiguous ideas and not having a clear vision in my experience.
    - BurningFrog 6y ago
      I'm always building something I've never built before. And the requirements always change as you go. So knowing what I'm building in advance never really happens.
- deleted 6y ago
  [deleted]
- Sammi 6y ago
  "Everyone has a plan, until they get hit" - Mike Tyson
- deleted 6y ago
  [deleted]
KKKKkkkk1 6y ago
Here's a nice quote from Jony Ive to support your point Much of the design process is a conversation, a back-and-forth as we walk around the tables and play with the models. [Steve Jobs] doesn't like to read complex drawings. He wants to see and feel a model. He's right. I get surprised when we make a model and then realize it's rubbish, even though based on CAD renderings it looked great. He loves coming in here because it's calm and gentle. It's a paradise if you're a visual person. There are no formal design reviews, so there are no huge decision points. Instead, we can make the decisions fluid. Since we iterate every day and never have dumb-ass presentations, we don't run into major disagreements.
- nikofeyn 6y ago
  i'm not sure ive has a lot of lessons to give on evolving an architecture in a way that improves maintainability and adds functionality. ive's entire MO revolved around removing those two things from the systems he designed.
arkh 6y ago
Have you heard of our lord and savior, Domain Driven Design? Here are the blue and red books.
- r4mbini 6y ago
  Are these books Evans(blue) and Vernon(red)?
  - arkh 6y ago
    Yes. I think Vernon's is the best as it has more emphasis on the process and organisation instead of the code.
    - r4mbini 6y ago
      Interesting. I’ve only read the blue one, will get a copy of red. Thank you!
logicchains 6y ago
>IME whether you rewrite or refactor, the lesson is the same: You have to grind your way into the good architecture. I think that only applies to things nobody on the team's built before. From my domain, HFT, if we hire a senior dev who's built blazing-fast futures arbitrage systems working for other companies, chances are if he rewrote our system it'd have a very nice architecture from the get-go, because he's already solved a similar problem multiple times before (and learned from the mistakes in those architectures).
- Someone 6y ago
  And then, that senior dev will pick a newer, better CPU, on-device bus, network topology, or your setup simply has a different scale, and hard-learned heuristic “don’t use more than x foo’s for y” or “the network is the bottleneck; use huge machines” become obsolete, and your dev will get some new battle scars. I guess that will happen in HFT all the time, as you don’t want that blazing-fast setup, you want something even faster.
constantine42 6y ago
"Grind" your way into good architecture? Good architecture is designing, not trial-and-errored. The only grinding I do is mental before I type a single letter of code.
- TheSpiceIsLife 6y ago
  So all the code you’ve ever written was perfect in every way from day one? You’ve never made a suboptimal choice? And never made a mistake?

readme 6y ago

Refactor is a good word to use if you don't want to upset the coworker who's code you completely replaced.

3pt14159 6y ago

I rewrote a codebase for a major stock exchange in about four months. It was originally in Ruby, which is my most proficient language, and my implementation was in Python, my second most proficient language. The original Ruby codebase had been worked on for years and was a complete mess. Not because of Ruby, but because the people that developed it were a little sloppy or junior or time pressured. Who knows? It's totally possible to do a rewrite and to do it well. The advantage of having a pre-existing implementation is that it's far, far easier to write tests before you implement because most edge-cases are covered in the existing implementation. Make sure you do a feature freeze before you start, then once you're ready for production do both shadow testing and canary testing and things will go well.

majormajor 6y ago
> The advantage of having a pre-existing implementation is that it's far, far easier to write tests before you implement because most edge-cases are covered in the existing implementation. Make sure you do a feature freeze before you start, then once you're ready for production do both shadow testing and canary testing and things will go well. There are two ways I've seen it go vastly over schedule when these things bite you. 1) There might be edge cases covered in the original implementation that neither you nor current stakeholders remember or know about. So discovery can either take way longer than expected (until you're confident you know 100% of what the code does) or implementation can get constantly delayed by "oh, yeah, we do still need that." 2) Feature freezes are a rare gift. Usually it's "we need this new thing" which leads to engineering saying "that would be hard to do in the current system, we should rewrite it" so now you have to rebuild all the old stuff plus build the new thing. And its requirements will probably keep changing on you.
deleted 6y ago
[deleted]
celim307 6y ago
God bless whoever wrote tests before you. Whether it be a rewrite or working on the current code base, those who have the time and resources and don't write tests are inexcusable and bad stewards

sys_64738 6y ago

When a developer says "I am going to rewrite ..." you have to question why? If you were not doing this "rewrite" then what else would you be doing? Refactor is usually a term used to minimally modify legacy code to allow new functionality to be added. Rewriting implies rewriting legacy code (regressions) and wasting time. If a developer says "it's the only way" then you need to be sure you can trust their judgement.

esperent 6y ago
I frequently write code that is publically available on GitHub and used by other people. Sometimes my first attempt at structuring the code works out well (more and more often as I grow in experience). Sometimes... Not so much. Usually a refactor is enough, but occasionally I have to buckle down and say, to be ready for public consumption, a rewrite would be faster.
to11mtm 6y ago
I tend to agree. Part of the problem is when developers get thrown into unfamiliar code bases but can't read code well, they may think about just doing a rewrite. But really, Thoughtful refactoring is better. And as another comment noted, in many cases good architecture comes from learning by past mistakes of yourself and others. OTOH, there are edge cases. I'm currently finishing up a rewrite, but it was due to the existing software being written in a language that most people aren't proficient in and isn't really the norm in our company. I've witnessed but did not participate in two rewrites. In one case, a company wanted to switch languages and databases (got sick of oracle licensing.) In another case, the company had a pretty good justification for a rewrite; the system was a monstrosity of .NET on top of VB6. In fact, in this case after the initial rewrite attempt failed and there was a change in management we pivoted strategy. Instead of trying to rewrite the world we would just continue to refactor things as we made changes to those modules, and only hard-rewrite the VB6 code. I don't know if they're still doing that, but it was a -lot- more productive than the rewrite team spending a year producing nothing that ever made it to production.
ydnaclementine 6y ago
> In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."

brilee 6y ago

This blog post doesn't seem to mention how long the process took - that's what I'm most interested in!

jt2190 6y ago
Yes, this is the key warning of Spolsky’s post [1]: The rewrite team had invested years of work and still had nothing shippable. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...

tjr 6y ago

My team inherited a 70%-complete AngularJS application, the user interface for a passengers and crew on a new series of airplanes. It worked fine enough on modern tablets and phones, but there were also some interface devices hardwired into the planes, which would be very challenging to update at this point, and on those devices the software ran very slow. Through several experiments, we determined that the sluggishness was unfortunately coming from AngularJS. We tried some performance improvements, but it wasn't getting very far. We built some prototypes using other Javascript frameworks, and found VueJS gave us the best performance. Management was concerned about us rewriting. The end customer was concerned about us rewriting. So risky! But we saw no other way to deliver adequate software performance. We got done on time, with great system performance, to the accolades of a very happy end customer.

momokoko 6y ago
I’m my career, UI rewrites have been By far the most successful. It may just be anecdotal, but my assumption has been that most of the edge cases are more easily identified because UIs are by nature going to have most of their business logic visible to the human eye.

k__ 6y ago

A good rewrite looks like a refactor. Strangle that monolith and none will be the wiser.

ratww 6y ago
I love that answer. Strangling for me is the best of both worlds: you can do it iteratively/cautiously but at the same time you can start "fresh" if necessary/possible.
teddyh 6y ago
Conversely, a good refactoring process eventually results in a complete effective rewrite.

WalterBright 6y ago

I constantly refactor my code, even the ancient stuff. Here's my most recent one: https://github.com/dlang/dmd/pull/11202 https://github.com/dlang/dmd/pull/11202 which changes a data structure from a linked list to an array.

mattkerle 6y ago
I was about to comment something sarcastic, then I googled you and instead I'll just say thanks!
- WalterBright 6y ago
  I don't mind if you say something sarcastic, but I am happy to say "you're welcome!"

pan69 6y ago

Maybe controversial but I think original authors should be able to rewrite, i.e. they can learn from previous mistakes and incorporate those lessons into the rewrite.

Cthulhu_ 6y ago
Oh yeah they definitely should, after all they will also have a lot of domain knowledge which new developers have to relearn (I'm in that situation myself at the moment). But, they need to have actually learned since then. They need to prove that if given another shot, they can do better. I've been able to do that, we built an application in BackboneJS, then Angular became big; we used that in a separate application, then decided to gradually rewrite the original in Angular (we were able to ship it to customers during the rewrite, rewriting things on a per-page basis without doing much with the design). It took a lot longer than the original, but also because new features were added and we were doing things much neater, with proper unit tests and the like. We learned that we had underestimated the complexity of a single page application, that it's not just a matter of "show whatever the back-end is returning so make sure the back-end is tested, front-end doesn't really matter much".

mamcx 6y ago

Refactoring/Rewrites is the MOST rewarding task that can be achieved with both legacy and new code. I do it from the start of my career (when I don't even know the name of it). And that include, regulary, rewriting to different langs (like, for example FoxPro -> C#, C# -> Python -> F# -> Swift (aborted) -> Rust (noob) -> Rust (now I know rust, almost!) <-- yes, the same project, years on it. Exits a lot of FUD against this. Is risky? Is hard? Can be messed up? Everything is staked against you? Yep LIKE ANY OTHER CODE ACTIVITY. ---- Things that help in make this a success (assuming, of course, an average, half-decent developer. But I have done it when I not even know anything, so who knows if it work?): CRITICAL: - Have source control - Have a task manager (mine: pivotal) - Split stuff in small tasks (couple hours max) - Priorize to know the data(schemas), the business REAL priorities (because all the time, everything will be declared of imperative importance). The code? Less so. If the code is good then you are not considering this at all. - This can be done in the small or in the large. So, a full rewrite can be totally done in hours. If all is so tangled this is not true, then the code is beyond salvation, REWRITE. Nice: - If possible, deployable code in one go (of the old code) - The database (or data in general) is kind of easy to grasp (if not, all is a mess, REWRITE). With a good database, rewrite is far more profitable than refactoring messy logic. - Have good test that can be used to validate it? Great. I rarely have that luxury. To become good: - Rewrite/Refactor is a muscle. The more you do it, the better you become. - Do A LOT of MICRO side-projects or experiments. Trash them, rewrite them, explore them. Make your mind used to it (note: this is not "ruin my life, only code yo!", is to start a major idea with a micro side project before commit bigly) Everyone know the second time your code is better. Imagine if you do 5 rewrite in successions. That code will be celestial! - Be confident in being able to read code. Read code. And read more. Much more if is a new lang. - Ask question, and ask the same question, differently. - Pick a project that you can do in your sleep (mine: Pseudo micro ORMS) and start with that when new lang or framework. In a side project. Must be of small size (few files, at most) - If change lang, framework, be aware that some are BETTER than others. Learn to know why and when start using it. - Dedicate a few minutes learning useful stuff. - Do the HARDEST stuff first, at least, a significant piece of it. If the hard is done, the rest is easier! - Have a solid selection of companion tools that you RARELY need to change (mine: Sqlite, Postgres, nginx, ubuntu (deploy)) The MORE messed up, the BIGGER is the payoff: - Not have tests or are all useless? Start writing test and all is tangled? Rewrite - Like a stupid idiot you are 4 months into a refactoring (doing stuff properly and all) and all paths lead to doom? Rewrite (you probably feel it 3 months ago. Idiot me) - You testing (manually, auto, whatever) take X times than write code? Rewrite - The customer is on fire, nothing work, all things are delayed? Rewrite - Everyone in the path of requirements are even more lost than you? Rewrite --- Surely exist some stuff that is conveniently ignored in this list: Have a customer/team that accept the rewrite. Look, if the customers/boss/team is THAT bad, then NOT MATTER what you do, you ARE SCREWD. But I'm lucky, and have worked in situations with serious trouble (but decent humans around, even if can't help much and I must figure a lot on my own) have tell me that people are more accepting of rewrites than some could think. Even delayed projects can be tolerated if exist a good flow of improvements. P.D: This is my experience in my sector, and the few I have done in a mid-size startup...

satyrnein 6y ago

So they got off of Elixir and microservices, but their introduction of Typescript and GraphQL didn't pay off like they hoped, and now they need to get off GraphQL. I think the lesson of "Choose boring technology" applies here. One of the reasons rewrites are risky is that very often the new tech doesn't live up to expectations.

bsaul 6y ago
yeap, this is a good point. This is the first time i see a rewrite to "older". Older architecture, older language, older methodology. And the only part that serm to not have met expectation are the ones using new stuffs ( graphql and maybe typescript). Maybe there's a rule here : complete rewrite is ok if you're rewriting to "older" tech.
devonkim 6y ago
If the business model is boring, you’ll need complex technology to compensate. If your business model is complex like in a lot of enterprise software, you should strive to keep the technology as simple as possible. Simple doesn’t mean to not use complex or cutting edge things IMO but to be conservative in using things that push the cognitive load of your collective team. A 10 person company probably doesn’t need a K8S cluster for a mobile app but I’d be surprised if there isn’t one for a 10k+ software company.
- jspash 6y ago
  "If the business model is boring, you’ll need complex technology to compensate." Could you elaborate on this? What is the complexity compensating for? Boredom of the developers?
  - ansgri 6y ago
    I suppose the boring business model needs some secret sauce, i.e. advanced technology, to stay competitive -- since it's boring, many people are likely to do it, the margins are thin, you need to optimize.
  - 6y ago

bdcravens 6y ago

I feel there's room for something in-between (reimplementation? resurrection?): start with a clean slate, and bringing in large swaths of functionality and code and then rebuilding other parts. I think this is useful when you have a lot of work to upgrade an older version of your core framework or many dependencies, you're changing a large piece of your infrastructure (like a database), or you have a lot of work to "fix" your test suite.

cryptonector 6y ago

Some rewrites can be easy because a better technology exists that makes it easy. For example, you might have a NoSQL DB with lots of hand-rolled queries, and then you get a query language for it, so now you can rewrite a great deal of code. You still hand-roll transactions because eventual consistency, CAP, and all that, but then you get a SQL DB that... gives you all of that anyways but with good ole SQL (with eventual consistency, not ACID, but hey, CAP), so now you can rewrite the queries and the transactions, and maybe that's the lion's share of the app, so you've essentially rewritten the app. What about the REST layer? Well, you get inspired by PostgREST and write a similar tool for your new DB and there's nothing left of the original.

Ericson2314 6y ago

I think everyone is missing the big point here: they current implementation was in an untyped language and so refactoring was just to hard. Use languages with good type systems and then refactoring is easy, and always the correct technical choice. Let me say that again, the killer app of type systems is that no mistake can "total" the code.

Chris_Newton 6y ago
More expressive type systems might make refactoring safer, but they don’t necessarily make it easier. The former happens because it’s harder to change something encoded in the types accidentally. For the latter, it should be easier to change something encoded in the types deliberately, but often the opposite is true.
- jillesvangurp 6y ago
  Having a type system enables zero risk automated refactorings, even complex ones and make them a no brainer. Not having that builds a reluctance to do even simple refactorings. A good example is the simplest possible refactoring: renaming things. I was doing this in pycharm on a simple python project the other day and it proposed modifying just about all dependencies on the classpath because it couldn't tell apart things that were in scope and out of scope of the refactoring. I've seen similar things happen on javascript and ruby codebases. Renaming things is a PITA in those languages. Not safe at all. On any Kotlin or Java code base I do this all the time without thinking twice. I rename stuff, I move stuff, extract variables, auto fix things, etc. It just happens. A rename is a complete non issue for that. Doesn't matter if it's a local variable or the package name of your entire code base. You can trivially modify thousands of lines of code with a keystroke without breaking stuff.
  - Chris_Newton 6y ago
    It seems to me that what you’re talking about there has more to do with having clear rules in the language for scope and modularity than to do with the type system.

nottsdev 6y ago

I think rewriting is a great option, provided both options are weighed up appropriately. However saying this, we made the decision to rewrite a legacy system around a year ago and the project got parked due to "urgent" projects being fed to our team. This was frustrating but we could not account for this external factor when weighing up the options.

gonzus 6y ago
I think changing priorities is a risk that has to be taken into account whenever you are considering any task that will extend for a long time -- rewriting code is just one example. If you have to go on for many weeks / months on a project without the possibility of showing intermediate results, it is naive not to think that there is a high risk of the project being cancelled.
- nottsdev 6y ago
  I completely agree with you, it is just a complicated metric to factor in. I guess one of the frustrations was the fact that our team had been been assigned with assurance that we will not be affected by changing priorities.

gampleman 6y ago

A really undervalues idea in software development is architecting software for rewriting it in the future. If you make your software small and well defined, then it becomes a reasonable investment to simply rewrite it from scratch. This is a great talk about this idea: https://vimeo.com/108441214 https://vimeo.com/108441214

Wildgoose 6y ago
Optimise for Deletion! I always say this, but it's difficult to get people to understand just how important a concept this is.

jillesvangurp 6y ago

It's always a judgment call. But in general, there are a few things that make the choice easy. Any UI project, I tend to look at the expected shelf life of the thing knowing full well that most UI projects get discarded pretty quickly. Stuff like backbone, angular 1, etc. were pretty popular a few years ago and you don't hear a lot about people whining about refactoring those. Mostly those code bases have already been changed; several times probably or abandoned. So, if we have a bit of UI and it's important for it to be right and it is clearly not and I have a team of developers leaning towards unceremoniously dumping the old code base, I'm not going to object very long or hard. The reality is that it probably can be fixed but all the people interested in doing that have gone. For backend systems it's different. I regularly see code bases with long histories. Often a good sign of how good they are is if people can work on them with confidence. In those cases, I lean towards doing more incremental changes. Update dependencies, modernize a few things even if they've been fine, etc. If that's not the case, it's a net cost sink and something needs to be done.

franciscop 6y ago

> a handful of engineers focused on machine learning (ML). [...] We leaned into a singular moderator talking to a group of people. I am curious about why an early chat/conference company needs so many ML Engineers? What are you using them for? (the Remesh mentioned in the article seems to be https://remesh.chat/ https://remesh.chat/ and not https://remesh.ai/ https://remesh.ai/, based on previous articles, the logo and blog domain)

albertzeyer 6y ago

I think Google even has a policy that almost all parts of code in their mono-repo needs to be rewritten (not just refactored) every 2 years or sth like that.

satyrnein 6y ago
I guess nobody felt like rewriting Google Reader!
- WJW 6y ago
  Not "significantly technically challenging" enough, so no promotion opportunities there.

cientifico 6y ago

In my experience, refactor vs rewrite is the wrong approach. Knowledge of the business must be maintained. If knowledge is on the team because the company is stable, rewrite. 37 signals is a clear example of this. If the knowledge is in the code, because multiple people or teams worked on the code base over the past few years, refactor. A rewrite will be the recipe for disaster. I never found an exception to this rule.

dagss 6y ago
I have been responsible for an exception to this rule. We were new people coming in to take over, decided to go for rewrite, and were successful. But part of the reason for the rewrite was that the logic (translation of business domain knowledge into code) in the old version was wrong resulting in unstable behavior. It could have been fixed in the old version (very many places) but the lack of a test suite too tipped the scales for rewrite (with test suite). You can say that we learned the domain (at least) as well as the authors of original system after coming in, but won't that be the case of any successful rewrite?..
- cientifico 6y ago
  If you are able to get the domain knowledge fast enough, sure. Or as you said, the domain knowledge has changed from the original.
- rakoo 6y ago
  It seems to me it's still not contradictory to GP: domain knowledge wasn't in the code, it was outside. The thing didn't work in all cases so you had to get the specs from something other than the code; rewriting according to this exterior source was the sensible thing to do
chosenbreed37 6y ago
Great heuristic!

Cthulhu_ 6y ago

I'm in the middle of a rewrite, but because my employer also wants me to do work on the existing application, I end up doing a lot of refactoring as well - if only for my own sanity and to help me understand the existing code. We're talking an only 8 year old (continuously developed during that time) app built in PHP and JS, but by someone who never bothered to learn web development proper. Resulting in a lot of copy / pasted code, string concatenation to produce HTML, etc. The problem isn't so much that, it's that the original developer was also highly productive, so the codebase exploded to nearly 200K LOC. They recognized the existing UI was not going to work, so after the previous developer left, they put out a job advert to which I was recruited for. The job being to rewrite the application with modern technologies, with stuff like structure and tests built in this time. I was okay with that, I mean I could probably do some things with the existing UI but it would be very labor intensive and if it was ever finished, it would still be a mid-2000's-looking Dojo app. I've (for now) decided on Go and React, deploying it as a self-contained application (our distribution is via RPM packages on bare metal or virtual servers). Not sure about Go at the moment, most work is straight data manipulation (think REST API, database and writing XML files), struggling a bit with things like data consistency (because Go won't enforce you to set all fields, mind you neither does Java but you can have Lombok generate constructors and the like). Anyway, my problem right now is that I was hired under the premise that maintenance of the old app would be like a day a week at most. Of course, in practice they're asking me to add new features to the existing application, so I've been spending over half my time on it instead, not making much headway with the rebuild. My fear right now is that they'll scrap the new UI in favor of just keeping the old one around. But there was an article a while ago that pointed things like that out, I believe it was titled something like "why rewrites fail". Very close to home. I mean I still believe a rewrite will be beneficial, but it's going to take at least another year and a half I think before it's feasible to replace the existing UI. Should have a chat with the boss whether we can hire more people to work on it and / or the existing application.

satyrnein 6y ago
If it hits the same database, build the new features in the new stack, and launch them. Users should click from the old to the new without even realizing. You have to figure out some details like sharing sessions, but this way avoids adding onto the old application. Otherwise you'll never catch up.

barking 6y ago

I don't know if this is off-topic or not. More times than I care to admit I've lost hours of coding due to crashing without saving first. Without exception when I redo the lost code it's far better designed than the original and usually (not always) in a lot less time. I'd almost be inclined to recommend always throwing away your first completed effort and rewriting it while your mind's fresh.

eddieoz 6y ago

Probably it should consider the alignment to the business strategy. Developers tend to rewrite because of tech challenges. But I agree with the conclusion and rewrite should be considered if it is impacting the strategy.

battery423 6y ago

I would argue that rewrite is slightly more difficult then refactor. Refactor means, after all, making the current system better while rewrite would mean no update/change/feature on the current system and more pressure to get it done. I would herby say that a good team with experienced people will succeed in both cases while an less experienced team will probably have a higher chance of doing a refactoring then a rewrite. I like refactorings better. It allows me to optimize a system slow and steady. I like rewriting better if you don't need to support the old env while doing the rewrite and having expierence in the old system.

fabian2k 6y ago

I found the move away from GraphQL again interesting, maybe because it supports some of my concerns about it. I never looked in detail into GraphQL, but just from the general principles I understand I found it rather scary to decide to use it. It's a big complicated thing directly between the data and the frontend. If anything goes wrong there, now I have to debug a much more complex piece of software than if I use a REST API. And just like in the article, based on my superficial knowledge I was concerned that it could be easy to create bad queries, and hard to fix performance issues. I'm not saying GraphQL is bad, I really don't know enough about it. But given the central position it takes, I would not feel comfortable deciding to use it without a serious amount of prototyping and getting to understand how it works. It often feels weird to me that such a complex piece in this kind of critical position is often casually recommended for any kind of web application.

emerongi 6y ago
> it could be easy to create bad queries, and hard to fix performance issues You can do complexity analysis of the query before you run it and deny requests that have complexity above a certain value. Some graphQL libraries provide this out of the box. Of course it won't eliminate all possibilities of bad queries, but at least most of them. For me GraphQL simplifies life quite a bit. We have an internal library that plugs a lot of things in automatically but leaves full customizability options, which makes it quite convenient to build APIs. On the front-end there are very useful libraries that again make life simpler. In short: the biggest benefit is that I have more time to spend on things that actually matter, instead of inventing my own protocol that in the end is just going to be a worse graphQL.

dirtylowprofile 6y ago

My last 3 clients wanted me to rewrite their React Native apps back to native iOS and Android like in a separate codebase. I’m not sure if this is a trend but their common complaint was that they are sick of the performance of the app and each platfrom has a unique bug. Also this one project written in RN took like a year to develop.

blackpanda 6y ago

I think we need to focus on even more basic point while deciding between refactor vs. rewrite. That is whether the basic DB schemas/Data Structures are well designed/suited to current business understanding or not. Because rest of the structure stands on top of them. I do not see how you can refactor stuff if the majority of the DB schema/Data Structures are built incorrectly and/or not suited to current understanding/layout of the business. You will have to rewrite. The only case where refactor can possibly work is that basic DB schema/Data Structures is 90% good and changes are only required at the business logic level.

code-faster 6y ago

Rewriting has a concave payoff. It requires high levels of certainty to be valuable. The bigger the rewrite, the more likely you're less certain than you think. Refactoring is the same as rewriting, just on a smaller scale, which increases certainty. Joel's advice assumes you will overestimate your certainty.

dep_b 6y ago

I hear more often things like "oh I don't have experience with X, so let's not use it". I mean programming languages are easy, frameworks are hard. Like learning how to use Django (which is pretty easy as far as frameworks go) to me seems harder than using React (assuming I know that already) with TypeScript instead of JavaScript. I would probably be really happy to use TypeScript instead. If Elixir was a really bad match because it was hard to use the required libraries, OK I can understand that.

commandlinefan 6y ago
> programming languages are easy, frameworks are hard That's been my observation as well, but I find myself surrounded by folks who insist that any framework should be a ten-minute tutorial away from mastery (even people who ought to know better).

aazaa 6y ago

> ... We had very poor test coverage in the areas most in need of refactoring because they were the oldest code, written before we established good test practices. That's probably not a coincidence. I like to ask those most strongly opposed to test-first development how they refactor code. I have yet to get an answer the leads to anything other than a rewrite at some point. Of course, the rewrite might be coming for the test-first codebase as well. But in that case, the test suite can be leveraged to great effect as a detailed specifications document. This is one of the benefits of test suites that doesn't get enough attention.

bearjaws 6y ago
How is this a test-first argument? You can write tests at anytime, bad discipline is bad discipline. I've never worked somewhere that didn't write tests because "they didn't write them first", they didn't write them because they were an immature organization. My current job leaves it up to the developers, and most don't write tests first. Its no miracle that despite us not writing tests first, all changes have their unit tests, integration tests, and end to end tests. This is due to the fact its baked into our SDLC and enforced in code review.
- commandlinefan 6y ago
  > You can write tests at anytime Actually, every single Java enterprise application I've worked on for the past 20 years has been test-resistant: the developers declare everything "public static" so none of the application can be run without running all of the rest of it (since each class imports another that transitively imports everything else, all with static initializers that read configuration files and connect to live database instances). Every refactor/rewrite I've ever suggested has been specifically to allow tests to be written.
ratww 6y ago
Wait, this is confusing. By test-first do you mean TDD? Even if you don't do TDD you gotta have tests before refactoring, I've never seen anyone writing tests after refactoring. I don't think this is controversial at all. Anyone please correct me if I'm wrong. (I've seen refactoring without testing but that's another matter)

namelessoracle 6y ago

AngularJS seems like the number one culprit to cause people to raise their hands and go with the rewrite over refactor approach.