6 ms·
Legacy Software Systems: How to Live with Aging Software Architecture?
- ravagat 4y ago> Software developers are often regarded as architects, as their work somewhat resembles that of a civil engineer. I like this analogy.
- marsven_422 4y ago[dead]
- mgl 4y ago#1 Keep track of the most important updates #2 Don’t rewrite what you don’t have to #3 Maintain good documentation, gather the necessary knowledge
- ramraj07 4y ago#2 is not universal. There’s a lot of legacy software with outrageous workarounds and sometimes it’s simpler to rewrite part of the system than to even maintain it for a month. It’s all about good judgement.
- grvdrm 4y agoFun question to think about what you have to rewrite vs not. This reminded me of my last firm where we had a C# system with tons of Task Parallel Library code, nearly all of it written poorly or not written for the correct use-cases. My colleague and I weren’t part of the development team responsible for putting it together but it was clear it has been haphazardly built that way over time. We were quite convinced the team needed to rewrite but then things like cost, time, priorities stepped in. We needed other areas addressed. So, the code stayer because it worked, enough, unfortunately.
- SoftTalker 4y agoThat’s often the first impression but in my career I’ve never seen a rewrite work substantially better or save any money. Sometimes they are necessary for other reasons but those two are the arguments usually made and they don’t really hold up.
- switchbak 4y agoI've got experiences in both camps. Well managed rewrites (typically portions of a system) can fix a lot of problems in one fell swoop, but you have to really know what you're doing and actually have concrete plans for improvements. If not, all you've done is just replaced a bad system with another bad system. I've also seen that lately. You're right that often it's just "different" and not better, and that makes it hard to sell the case when a rewrite is truly needed.
- jamesfinlayson 4y agoAgreed - I've worked with one rewrite from scratch and it convinced me that rewriting from scratch is never a good idea. Incidentally, I can see why the company decided on a rewrite from scratch (no separation of MVC logic, no framework, almost no tests, only ran on special environments, production had an extra set of patches that for some reason would never get merged back to master) - honestly, it really was too far gone.
- indymike 4y ago> it’s simpler to rewrite part of the system than to even maintain it for a month. This is often a career limiting decision. Rewrites are often much more difficult, particularly when you are dealing with a system that is not fully understood. This often leads to what should be a quick re-write taking > 3x more time and money than projected. In the end no one will remember why the technical reason for the rewrite, but they will remember who made the decision. I've made this mistake a few times in my career, and I always thought, this project, it is the exception to the no-rewrites rule.
- layer8 4y agoThe trick is to make someone else make the decision.
- anonuser123456 4y agoUntil you find out there is some essential complexity you missed that’s encoded in the convoluted logic… and then your rewrite doesn’t function and you look foolish.
- layer8 4y agoDepending on the case, it at least has the benefit that the essential complexity is now known and can be documented.
- goto11 4y agoYes but before you can make that call you need to have a reliable estimate of how costly it will be to rewrite the system. This in turn requires a through understanding of the current behavior of the system, which means you will have to study and understand the code anyway.
- Dalewyn 4y agoRule 0 in Engineering: If it ain't broke, don't fix it.
- Buttons840 4y ago80% of the time the answer is "find out if the owners of the code are supportive of improving the code", and when the answer is "no", just put in your hours and collect a paycheck.
- leidenfrost 4y agoDunno, I successfully refactored some legacy services. Some of them even were a big index.js spaghetti files. You have to start with the smallest and most conservative to the most "refactory" ones: - figure out what enters and what exits from the program, and create a test suite that reflects every known combination - move parts of main() to funcions - figure out what variables can be converted from mutations to returned values - move thr ifs(){} to maps, early returns and such - figure out the database schemas from newly written data - infer model relationships fro further writes - attempt to implement some pattern design, even if its just dummy variables Sometimes you even have to do it at the last stretch of time of your workday, and slowly the mess will become clearer, your test suit more exhaustive and comprehensive. And at some point it will achieve feature parity with the legacy service
- mixedCase 4y agoThe GP is referring to the political costs of changing bad code, not the technical challenges which are almost always manageable by any experienced programmer. Egos, however.
- leidenfrost 4y agoYeah. But there were times where I did some hidden refactoring even if people said "no" if I asked for permission. In part to maintain sanity in a toxic environment, and in part because I knew they just accepted it if it was handed to them.
- eska 4y agoYou’re the guy in corollary 1 in this article. https://yosefk.com/blog/people-can-read-their-managers-mind.html https://yosefk.com/blog/people-can-read-their-managers-mind.... Either you’re successful AND your managers have an epiphany that you were right and they were wrong, or you fail and your managers will punish you for ignoring their instructions. It’s a risky strategy for an employee.
- luciusdomitius 4y agoWell, the car analogy is very good I think. A 35-year old Mercedes-Benz W124 that has been taken care of during its entire lifetime is actually usable and could be reliably maintained. A 10-year old Hyundai with 50k km old oil is meant to be scrapped though. In the same way, there is legacy software and legacy software. In a decade of contracting I have seen both pieces of crap with 50%+ of dead code which are inmodifiable beyond adding/removing fields from some CRUD entity, and 15+ year old J2EE competently designed, maintained and documented systems, where the only difference to a fancy new project is that there is xdoclet instead of annotations and a ton of plumbing code around. The big difference is in telling them apart in advance - with cars you know that some brands are more maintainable than others as you see them on the road and as all of them share the same components/weak spots it doesn't take long to figure out if it is an oldtimer or a piece of junk. With software it is much more complicated.
- alexott 4y ago10 years ago I’ve attended a talk from one of German car producer - they talked about 25+ years of software maintenance. You invest into tools 5 years before release to market, and they should work 20 years after release. So companies need to plan the whole cycle not only code maintenance
- nijave 4y agoIt goes all the way down the stack, too. I talked to someone working on CI at Nvidia and they were testing up build toolchain to ensure some of their hardware accelerated libraries could be rebuilt in 10+ years (for automotive industry customers). The work mostly involved dynamic systems for ensuring the builds worked without an internet connection (monitoring network activity to make sure everything was cached/saved correctly).
- cpeterso 4y ago> The work mostly involved dynamic systems for ensuring the builds worked without an internet connection (monitoring network activity to make sure everything was cached/saved correctly). Documenting and archiving your build tools is also important. I knew a team that checked everything into source control, even Visual Studio. This ensures the tools were all preserved, everyone is using the same versions, and the code is synchronized with the tool versions.
- GartzenDeHaes 4y agoHaving been involved in 3+ $8-figure legacy replacement projects, this article seems to be missing business, management, and cost perspectives. In particular, "non-compliant with recent standards" is the least important reason to consider rewriting a working system. Often, chasing whatever is popular at the moment is the root cause of maintenance problems in the first place.
- kevinmhickey 4y agoWish I could upvote more than once. What I call "Resume Driven Development" creates fractured architectures and reduces quality in stable systems for the sake of "the latest standard". Not the best spend for the business or users/customers.
- watwut 4y agoYeah, but on the other hand the "we ignore the resume thing" makes it increasingly hard to hire and keep people. Because when tech becomes too much legacy, developers can not afford to stay there. Your employees will be selected for more passive bunch. Or you have to pay more to keep them. And I do work on project exactly like that. I am paid more then would be market rate for same position and everyone is scared we (as in seniors) are going to leave. It has advantages, not just in pay, definitely. But some of these advantages are disadvantages for the company to be frank ... like that the whole environment gets biased toward more passive and sort of sleepy. (If you are non sleepy you get pay raise, they are trying to reward it, but the force of the system is strong.) It does not result in the best spend for customer or a company.
- cratermoon 4y agoOn the flipside, when your website only works with Internet Explorer 11, forcing your customers to keep an unsupported browser around, it's time to reconsider the aversion to keeping up with changes in the technology. It's doubly bad when your website is designed for software embedded in kiosks and the like.
- nucleardog 4y ago
- jt2190 4y agoI feel like I’m reading a rough draft, not a deeply thought out piece of writing. There are some nuggets of good sounding information in here, but the author doesn’t really go very deep. Chad Fowler had an interesting slide in one of his talks (circa 2009 I think) where he had a graph of something like frequency of errors produced by working code versus number of changes to that code. Code could end up in one of four quadrants: - frequent changes, lots of errors: suggesting that the domain was still being discovered or there were quality issues - frequent changes, few errors: suggesting the domain was changing or growing but quality was good - few changes, lots of errors: suggesting that the code was not adequately maintained or even understood, perhaps poor quality as well - few changes, few errors: this is the “legacy” code that works hard and costs less to keep running Clearly it’s a good thing to have that “legacy” code, by this definition, but I remember that he listed some downsides as well, but not the downsides that the junior dev who wants to rewrite then whole system comes up with. Edit: I think it might have been "Measuring & Analyzing Things That Matter When You Have Too Many Things To Keep Track Of" At 36:00, he talks about a ruby gem called for Rails "Turbulence" that measures code complexity against number of modifications https://vimeo.com/38252887#t=2160s https://vimeo.com/38252887#t=2160s He also references this article by Michael Feathers: "Getting Empirical about Refactoring" https://www.stickyminds.com/article/getting-empirical-about-refactoring https://www.stickyminds.com/article/getting-empirical-about-...
- cratermoon 4y agoThe best source for this kind of emprical measurement of code complexity is Your Code as a Crime Scene by Adam Tornhill. See also http://www.adamtornhill.com/code/codemaat.htm http://www.adamtornhill.com/code/codemaat.htm
- jt2190 4y agoVery cool! It looks like that's evolved into a company called CodeScene. Interesting demo by Adam: https://youtu.be/SWFwPkgLcpo?t=10 https://youtu.be/SWFwPkgLcpo?t=10
- miohtama 4y agoThe legacy software works very well in a static world: when nothing ever changes there is no need to update the software. Of course, this is not the case. The problems arise when the software updates are not budgeted. Especially the finance sector is notorious for underinvesting in R&D and instead of spending money on stock buybacks - stockholders appreciate. However this cycle is broken when the world actually changes. If there is no proper maintenance, usually in the form of software R&D, the legacy software change cost has gone prohibitive high and the parent company is unable to do anything. This change is often Schumpeter's creative destruction. There is a new technology. A new startup raises as a competitor, using more efficient software where they can do changes, and execute their core business, with much lower cost, outpacing the established competitors. Some industrial, like early mentioned finance, are however very-well protected due to barries of entry like the cost of initial capital and regulation. Thus, you are going to see more legacy software in industries where there is never need for a change.
- nickpeterson 4y agoThe thing about never doing that big refactor, is your team loses the ability to do it. Only maintaining legacy codebases results in people who can’t build things.
- lll-o-lll 4y agoThis is a big, big company problem in my opinion. If you have a dev force mostly just adding to an existing system, they never learn design skills. Those that built the system inevitably leave over time, and now you’re stuck unable to attract talent because “old tech stack”, and without staff that could build something new. Don’t stay at a company that’s gotten into that state. If your young, try and gain a mix of experience so you can actually gain those design skills.
- tangjurine 4y agoWhat if you're on a team like this right now but the job market is bad...
- deleted 4y ago[deleted]
- layer8 4y agoShop around anyway. It may turn out less bad than you think.
- bberrry 4y agoIf you start a rewrite or a large refactor, for Gods sake, follow through and _finish_. Otherwise you have made things even worse. I can't tell you how often I've come across the remnants of such projects, abandoned with some sliver left. I make a point to never leave it like that.
- cratermoon 4y agoFew places I've worked ever put improving the system in the plan. It's almost always said that "we'll work on it later, when we have time". Later almost never comes, of course. On the rare cases when there is a little time, the team will start something but have to abandon it when inevitably the business needs change and they get drawn to other work. Remnants of incomplete attempts speckle the codebase like bird droppings on a car parked under a tree for 6 months.
- bberrry 4y agoI think that's part of our responsibility as engineers to ensure that technical debt doesn't pile up. Business will want to focus on pushing new stuff and engineers have to be a counterbalancing force. It's literally your job. This hasn't been a problem in the places I've worked. Perhaps I'm lucky or it's the fact that I don't live in a very hierarchical society (Sweden).
- cratermoon 4y agoYou're correct that it's the engineers need to be responsible, starting with not offering lowball estimates for development and then taking shortcuts and ignoring problems just to get the features out the door. There are plenty of businesses perfectly happy with hiring developers that will ship things as fast as possible over developers that will take responsibility for the code. I've worked both places. Unsurprisingly, I don't stay too long at places where I find myself working with a team full of shortcutters.
- jamesfinlayson 4y agoYes I remember working on a legacy system (PHP application started in 2003) - there were two or three different attempts to make it MVC sprinkled throughout the codebase - obviously a few people had come and decided to improve things (and decided the last guy's work was no good) but left before making any meaningful progress.
- chrisweekly 4y agoIn this space I highly recommend the book "Kill It With Fire".
- worthless-trash 4y agoJust bought the audiobook. Looks okay so far.
- cpeterso 4y ago"Kill It With Fire" is great. Here's a 2021 thread about a review of the book: https://news.ycombinator.com/item?id=28571931 https://news.ycombinator.com/item?id=28571931 And two presentations from the author, Marianne Bellotti "We Killed These Things With Fire | Øredev 2019" https://www.youtube.com/watch?v=XoEfV0kXXDY https://www.youtube.com/watch?v=XoEfV0kXXDY "Future Proofing Systems with Marianne Bellotti" (2020) https://www.youtube.com/watch?v=_ICzo5r-7vY https://www.youtube.com/watch?v=_ICzo5r-7vY
- gonzo41 4y agoWhen the b-52 aircraft retires it will have been flying for about 100 years. I can't think of software system that has the same longevity. Maybe a credit card mainframe written in cobol. But what JS developer is going to learn that these days. I think a problem for software as an industry in it's infancy is that we've built a lot of stuff, and it's really easy to build new stuff, but it's hard to just keep the trains running on time. No one wants to pay for that. So you take the team that delivered the system and move them on. Or you stop them hiring and they age out of the product. The ghost plane fly's on for a long time untill you start to see mountains and need to change course. Then it's panic stations. Java cracks out a new release every 6? months now. There's a lot of java 8 out there. And it's not going to get upgraded without costly effort. btw, I have no solutions. I just think we're still at the early stage of this problem. And in truth in the built world. people rip down good buildings to rebuild shiny new ones all the time.
- bzzzt 4y ago> There's a lot of java 8 out there. And it's not going to get upgraded without costly effort. There's a Java LTS version for people that don't want to upgrade every 6 months. Almost all Java 8 code will run on Java 17 LTS so as a developer I don't see that 'costly effort' you're talking about. It's just build, fix and test as usual.
- jamesfinlayson 4y agoIs this what it's written in: https://en.wikipedia.org/wiki/JOVIAL https://en.wikipedia.org/wiki/JOVIAL? Maybe not all of it but some of it at least I'm thinking. I haven't been programming for long enough to see it or maybe it changes too slowly to notice, but I feel like the new programming languages of the last 15 years have been similar enough (at a high level) while the programming languages of 50 or 60 years ago are vastly different. If that's the case, maybe some of today's software has a better chance of lasting longer.