7 ms·
Rewrite, refactor, or reinvent? Lessons from 6 software rewrite stories (2019)
- thunderbong 4y agoExtremely well written article, building off of Joel Spolsky's original article of 'Things you should never do' [0]. When you should rewrite software from scratch and what should be kept in mind doing it. [0]: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- _fat_santa 4y agoI can't believe that article was written in 2000, it's aged so well and the lessons can be applied over and over.
- shoobs 4y agoThese stories are great, and totally resonate with me at the moment. I'm managing a team working on a 30 year old codebase. It is in technical debt, like many codebases that old. That being said, the codebase is full of so much good stuff from hard learned lessons over 30 years, and that's where its value lies. I'm convinced that the way forward is not to refactor or rewrite it, but to produce something new and different that draws on the 30 years of experience learned during the development of the original product. And, like Basecamp, run them side-by-side and allow users to chose what they want.
- kqr 4y ago> produce something new and different that draws on the 30 years of experience learned during the development of the original product. Are the original authors still there or do you have insanely good documentation?
- shoobs 4y agoMost of the key authors are still with the team, though some of the old crew will start retiring shortly.
- sohnakukkar 4y agoWhat is the domain and tech stack of this project?
- ebiester 4y agoI'll add another story. I think the hardest conversation circles around language choice. One problem with choosing innovative languages is that you can end up on a dead end. Zipcar, for example, used Tcl. While I am a fan of Tcl, it became obvious around 2012ish (IIRC) that this was not the future. There was a first attempt at a migration using rails, but that ended up being ditched for a JVM stack. (I don't know if most of it is still in groovy, but there were also pieces in ruby, kotlin, and java.) We also had a large number of performance and billing problems we needed to solve, and as we were guaranteeing compatibility and interoperability (a big bang switchover was too risky for many reasons), we had a lot of performance and usability issues for years. It cost far more than management expected in the outset for many reasons, but at the same time, staying on Tcl was unsustainable for developer recruitment and retention purposes. It's something that biases me toward blub languages now as a manager unless you have a very good reason to innovate there.
- jt2190 4y ago> … staying on Tcl was unsustainable for developer recruitment and retention purposes… I agree that this is a real phenomenon, and I find fascinating because it illustrates the role of fashion in software development. TCL may have been quirky compared to other languages, but actual working software was successfully developed in it by more-or-less typical developers. Yet it failed anyway because it wasn’t “cool”. Not being trendy meant that fewer brains were thinking about it, evolving it by improving tooling and writing libraries, producing training materials and evangelizing it. Ultimately it just stagnates.
- ebiester 4y agoI don't even know if it's "fashion" as much as an industry that only knew that java 1.2 wasn't going to be the path forward if you wanted to develop with speed. There was no guarantee python or php were going to be the winners. I think it does speak to the amount of labor that goes into making a language viable as an option, though. Java had paid labor. Tcl didn't have enough unpaid labor to succeed while python did.
- cushychicken 4y ago
- pjbster 4y agoGreat article. I think the key differentiator in all of these stories is the "stickiness" of their customers. Netscape's customers could switch at no cost (apart from migrating bookmarks, maybe?) and did. Basecamp's were much more invested in the product and 37Signals never abandoned them during the time they starting building its replacement. I get the impression (having never used it) that Fogbugz's users liked it but that Fog Creek couldn't win new larger customers once Jira arrived on the scene. And since it was a shrink-wrapped one-time purchase FC couldn't fund a paydown of their tech debt. Netscape will forever be remember though for this: "Less than two years after it was released, the company’s $3-billion IPO launched the dot-com era." This. This is why many of us are here reading this today.
- pocketarc 4y agoIt's interesting to now be at a point where we see a generation of SaaS make way for the next generation of SaaS. I've come to realise that realistically, platforms -do- change, and eventually you will need to either commit to a rewrite (even if you choose to do it by spinning up a whole new company like Freshbooks did). As an example, a SaaS app from 2010 might have been barely responsive, might have been put together with jQuery, and the backend might've been written in a framework that died off in the meantime. And while you can do a lot of little improvements and refactors here and there, ultimately the foundation on which you've built your system gets to a point where it slows you down more than it benefits you, and there's no real way forward. The stories in the article make me think that the old adage of "never rewrite" is really more like "eventually you will either rewrite or die".
- blowski 4y agoThe old adage is more “never do a big bang rewrite”, where the launch of v2 is one big step. An incremental rewrite, where perhaps you change a single component or bit of the UI, is always necessary. Although even there people moan about “change fatigue”.
- deleted 4y ago[deleted]
- Joeri 4y agoIn a large enough system a full rewrite becomes impractical anyway, because the team is sized to incrementally grow not replace whole cloth. The team simply isn’t capable of rewriting the whole codebase in one big bang release in anything less than years. You can bring up a larger team for the rebuild but that introduces its own risk, or you can leave customers out in the cold for years, like netscape did. Continuous rewrites are the only way to handle that kind of system, just like in a large enough city just to keep up with natural decay it becomes necessary to always have construction and road works projects going on.
- ojame 4y agoI’m a Tech Lead for a SaaS product that’s been around since ~2010. Four or so years ago we stared piecemeal transitioning away from jQuery/Backbone to React. Nearly all of our main UI is React now, but it’s still pretty coupled to Backbone. There’s a lot more risk removing that, so we’ve not had enough of a reason to yet. Basically the high touch (from a developer point) areas of the app are always the most modern and the first place to get whatever treatment we transition to (React, typescript etc) and there’s some really untouched areas that might remain prehistoric for years, and that’s fine. Its worked pretty well for us, and I can’t see why it wouldn’t continue that way.
- pprotas 4y agoGreat article! Something I was wondering about when reading this is how these lessons could apply to software that depends on its userbase. As an example, let’s use Discord; what if in 10 years the Discord codebase is seen as obsolete and requires a re-write? You can’t really make a “Discord 2”, because now you split up your userbase between Discord 1 and Discord 2. In this re-write you would have to make sure that you aren’t splitting up the community that you rely on, since it is part of the product offering (it is basically a social medium).
- samsquire 4y agoI think the question reveals a secret that nobody talks about. Your software is not understandable to others, so it's impossible to adapt. Your software needs high quality documentation about the mental models used to create and think of the software. But too many developers think the code is the documentation, meaning their code is legacy as written.
- jostiniane 4y agocan't agree more as somebody who joined a place for the sole purpose of maintaining already profitable products and continuously pay the tech debt there. I keep emphasizing it, looking at this project is like looking at a complied code, you can understand what this portion does by reading the program but You can't understand what the dev/architect had in mind when they did it, architectural diagrams + proper high level docs can do wonders. I can understand what those modules are doing, but it will take me years before I can grasp the entire system to know where to look for every fix/new increment.
- drowsspa 4y agoThat is so, so true. So many open source software that I'd like to be able to contribute to, if only they would put a Markdown file documenting at a high level how the code is organized, how data flows through it... Instead all we have is the code itself, sometimes some instructions for running the tests and that's it. Then every contributor needs to go through all of the code and reverse engineer it to even start.
- marginalia_nu 4y agoI briefly worked in the (Swedish) public sector, was on a team tasked to replace an old system written partially in Cobol and partially in some IBM-tech nobody understood anymore. The system was huge and poorly understood, and a lot of people used it to do most of their work. Extremely critical system for the very fabric of society. There had been several previous attempts at replacing it. They'd sort of fizzled out along the way, so they were stuck with several half-finished iterations of replacements that were in used in parallel with the old system. This was it though. Hundreds of people. Micro services. Modern JS frameworks. C4 architecture and hexagonal design and cloud native and every goddamn buzzword that's appeared in a programming blog the last 10 years. Just get it all in there! Everything the cool start-up kids were using needed to be apart of this. It wasn't clear exactly how it was going to better, but it sure was gonna be modern. I didn't see the project through but I could see where it was heading clear as day. It was only a year in and maintenance was already becoming a problem. Shit was falling apart. It was a nightmare to debug. Nobody knew how the heck all the services quite communicated. I'll eat my shoe if they don't spend the next 25 years fixing leaky pipes with this microservice mess bolted onto the previous iterations as an even more grotesque programming-cronenberg.
- xorcist 4y agoThis is a telling example of the innovation budget idea! If you're going to replace a creaky old legacy system, never ever do it with the latest technology. Do it with tried and boring technology. Otherwise soon enough you will have two problems.
- marginalia_nu 4y agoI think part of the problem was they needed to pitch the project to the government to get it funded. Probably easier to sell the idea of the Homer Simpson Model S[1] than what they actually needed. That, and everyone with any sort of deciding power were technology-illiterate bureaucrats who waste no opportunity to try to make us somehow use AI to somehow enhance this system in some fashion, largely so they could announce to the world that their system use AI somehow. [1] https://imgur.com/97lhWVR https://imgur.com/97lhWVR
- meekaaku 4y agoOne of the great software rewrite battles that I know of is computer graphics sector. It was mid-90s and Silicone Graphics (it hasnt rebranded to SGI yet) was the dominant hardware, and Wintel were too slow. In hollywood the flagship products were Alias PowerAnimator, Wavefront Visualiser, Thomson Digital Exploer, Softimage 3D for animation, Sidefx Prisms for particles/effects. PowerAnimator had the best modeling tools like NURBS. Softimage had best animation tools. None of these ran on Windows and all were running on SGI's own IRIX. Microsoft bought Softimage in 94 and ported it to Windows NT. Suddenly Silicon Graphics saw Wintel as serious threat. So they bought and merged Alias,Wavefront, TDI to form Alias|Wavefront and began to write Maya which would combine the technologies from all three. Softimage also started to write its new one codenamed at that time Sumatra which would eventually release as XSI. The race was on, and the two big giants SGI and Microsoft were having a proxy war now. Everyone was keen on seeing who is going to come out first and best product. Meanwhile Sidefx was also writing something new from scratc which they eventually released as Houdini in Dec 96. Sidefx has always been a smaller niche player and has stayed true to that till today. Maya 1.0 came out in Feb 98 and it was immense success. A lot of hollywood studios migrated to Maya because Sumatra was nowhwere near release. Eventually XSI came out around 2000 I think and it was too late, but not too little. XSI still had more powerful animation capabilities but they had lost a lot of ground to Maya interms of mindshare and customers. Eventually Microsoft sold Softimage/XSI to Avid. In the mean time 3D Studio from Autodesk was not a serious player when it came to hollywood stuff. So Autodesk bought Softimage from Avid. Unfortunately SGI also had real financial trouble due to Intel finally catching up interms of speed. They also sold Alias|Wavefront to Autodesk. So now Autodesk had the two major 3D graphics packages used in hollywood. Autodesk eventually killed XSI and continued Maya at their high end, and 3D Studio Max as their lower/medium end product.
- lelanthran 4y ago> Silicone Graphics No 'e'. Silicon is for computers, Silicone[1] is for breast enlargements. [1] Insert jokes about Silicone Valley here.
- herodoturtle 4y ago> Insert jokes about Silicone Valley here. Vesting.
- mch82 4y agoI recall reading an article about one of the database engines (maybe MySQL or MariaDB) and how it was created as a drop in replacement using test driven development. Can’t find it… anyone know of an article like that?
- thatwasunusual 4y agoMariaDB, probably. Testing didn't exist back in the days when MySQL was created. ;)
- jimnotgym 4y ago> If software is hard to change, it is your fault. That is a bit harsh on the new team that just inherited someone else's mess and the One Guy who understood it is not there any more.
- onos 4y agoI personally don’t have a great handle on how to reason about anticipated maintenance costs for a given project, yet would like to. Some simple notions: core libraries should be written with few dependencies so as to reduce update impetus surface area, simplicity of method is often better. But that’s as far as I’ve got. If building something to last (the dream) probably would be better to solidify this further.
- gwnywg 4y agoA few times in my career I have been in situation where decision to rewrite was made. Worst case was where decision was made, resources had been invested and then it was an ongoing theme for years to come (this particular company I have in mind is still talking about next version 6 years after initial resources were invested, I was the one to build foundation system which works to date even though I left 4 years ago) Best case was in the company I was contracting for, they had huge and almost 20 years old pearl codebase which they decided to replace with python ecosystem. In 6 months the company was already actively switching to the new system, exposing bugs early etc. In my view each time management is where it can result in good or bad outcome.
- tinco 4y agoI really wonder Spolsky didn't just do what Golang did and use the transpiled code as a basis for the continued work. Of course this predates the success of Golang but it's not a very far fetched idea, I bet Jeff Atwood would've suggested it. Of course happy they didn't because if they did Fogbugz would've had an incremental upgrade path on PHP all the way into modernity and would have been an acceptable thing to work on, preventing Trello from ever existing most likely.
- theptip 4y agoOne thing to note is that you absolutely can fully replace a legacy codebase a piece at a time, the big risk is when you have a completely incompatible “rewritten version” of the product that you will cut over to at some later time. If you do something like build a facade in front of the old service, and gradually move functionality into the new layer, this can be seamless for users. But of course, this doesn’t give you the ability to revisit product functionality, like the Basecamp rewrites. I think those are really great examples of how to continue to innovate and experiment, while keeping your old customers happy, and the key point there would be “rewrites can be done safely if you commit to keeping older versions around”. Basically treating the versions as distinct products, instead of the rewrite replacing the old version.