21 ms·

How to Improve a Legacy Codebase

• _virtu 9y ago
  How does one get better if they only ever work in code bases that are steaming piles of manure? So far I've worked at two places and the code bases have been in this state to an extreme. I feel like I've been in this mode since the very beginning of my career and am worried that my skill growth has been negatively impacted by this. I work on my own side projects, read lots of other people's code on github and am always looking to improve myself in my craft outside of work, but I worry it's not enough.
  • catwell 9y ago
    You can certainly improve some of your skills working on terrible code bases. For instance, you should become much better at debugging. You will have to learn debugging techniques and tools that you may never have had to use in other code bases. Also, here is a paradox: take someone who has only ever seen terrible code bases and someone who has only ever seen very good code bases. How can they know? They might take a guess based on how well the software works, but that's probably not very reliable. I think a good software engineer is someone who has seen a lot of different things, good and bad; someone who knows what design choices work and what will plunge software into the depths of Hell; probably someone who has make mistakes themselves and lived through the consequences. But yeah, when working on such a code base, do read some code outside of it now and then, never forget there are better ways to do things. And if you are starting to feel burnt out by the quality of the code base you work on, you should probably make a change.
  • artursapek 9y ago
    Start your own company? But even that I think is futile. The causes of manure code are usually out of your control - tight deadlines; new devs touching stuff without properly understanding the whole; organization prioritizing short-term reward over long-term sustainability. You also have to consider the inherent survivorship bias - only successful businesses live long enough that their codebase has time to grow into a big mess. Any company that lives more than a few years inevitably ends up with "manure". You'd have to be in the extremely rare position where you are profitable and have no pressure to keep growing (investors) in order to invest enough time into technical craft to not end up with manure code.
  • thehardsphere 9y ago
    You can learn a lot from mopping up steaming piles of manure. Recognizing what manure is and the thought processes/business incentives that produce it will be helpful to you in not making your own. Also, even if your current codebases are manure, that doesn't mean everyone in your company makes manure. Find people on your team who don't write it, and learn from them. If nobody is like that in your company, then maybe you should change jobs if you've been there more than two years. Cleaning up manure helps with interviewing because you can share your war stories with the interviewer.
  • FLGMwt 9y ago
    It sounds like you at least have a good feel for what's bad and what's worse (which is good). I think one thing you can do is attempt to isolate the code surrounding the next chunk you work on. Do as much as you reasonably can of the things the article mentions. This may only be writing tests and adding logging, but if it's an improvement over what's there, you'll improve the experience of the next person involved with that code. I'd warn you against jumping ship in hopes of finding a "clean" code base. Most code is somewhere on a spectrum of "maintainable enough" and something... grimmer. If you really are unhappy and don't feel like you're growing or have the ability to grow, maybe try out contributing to a well-maintained OSS project. If you find yourself immensely happier, dust off your resume ;)
  • deleted 9y ago
    [deleted]
  • cafard 9y ago
    Frankly, I never quite understood the importance of clear documentation until I found one such code base smoldering on my porch.
    • chiph 9y ago
      At the very least, write a doc that explains how to build the product, including where to find the parts in source control, what the dependencies are, what servers it'll get installed on, and so on. The goal being to increase your shop's "Bus Factor" https://en.wikipedia.org/wiki/Bus_factor https://en.wikipedia.org/wiki/Bus_factor
      • _pmf_ 9y ago
        > At the very least, write a doc that explains how to build the product, including where to find the parts in source control, what the dependencies are, what servers it'll get installed on, and so on. ... in the form of a Jenkins build configuration. (If possible; if the system requires legacy compilers that only run on old Windows versions or a proprietary compiler for an embedded target, good luck.)
        • chiph 9y ago
          The legacy projects are the ones where a doc with all that info would be the most useful. :)
      • _virtu 9y ago
        I love the notion of bus factor. Whenever a bunch of devs go out drinking I think this every time we cross the street. :)
      • Roboprog 9y ago
        Something to be cognizant of when creating this "keystone" doc - not losing it on some "share" that nobody can find anymore. Thus, the use of README in the root directory of a project.
        • 9y ago
  • xemdetia 9y ago
    I generally clear my head by reading mailinglists and looking at how projects of my interests do things and keep their commits in order, especially around bugfixes. OpenBSD is a fun one to read through as well as others. I also go to/watch talks about people managing their own piles of manure and change processes. As long as you keep your eyes open to other people doing what your organization is struggling with right the first time it gives sufficient motivation to approach every problem with 'why is this here and how could we do this better.' The great thing about the state of F/OSS right now is that you have codebases that have to change because of things like large amounts of RAM being so cheap- that very well understood algorithm designed to only do things in 64MB so as to not swap out no longer makes sense and so there are intelligent motions to fix it. I've been planning on reading the Postgres 9.6 changes for parallel queries to understand how they did the magic in a sane and controlled manner and shipped a working feature.
    • anarazel 9y ago
      > I've been planning on reading the Postgres 9.6 changes for parallel queries to understand how they did the magic in a sane and controlled manner and shipped a working feature. Very incrementally - we've been adding more and more infrastructure since PostgreSQL 9.4. Which finally was user visible with some basic parallelism in 9.6, which'll be greatly expanded in 10. There's some things that we'd have done differently if we'd started in a green field, that we had to less optimally to avoid breaking the world...
    • _virtu 9y ago
      Thanks for the response, this makes me feel a bit more positive.
  • mattnewton 9y ago
    While starting out, knowing what not to do and precisely why is nearly as important as knowing what worked. In the case of good codebases and bad codebases though, you still need to be careful not to cargo-cult wholesale the architecture that worked before, and conversely not to do everything different from the last horror you worked in. View it as a learning opportunity as you debug: some things they will have gotten right, sounds like many things have been gotten wrong, but the process of reasoning them apart is still valuable. All that being said, certainly do not hesistate to look around if you feel like you aren't growing as fast as you could be. Life is short and it's a sellers market for engineer labor in most places I have seen.
    • _virtu 9y ago
      Yeah, right now I haven't worked at my current place long enough to leave. I want to put in enough effort on my part to warrant feeling like I haven't grown enough too. I've decided that the best thing I can do is focus on what I'm not doing well enough or consistently enough until I feel like I've covered all my bases/can't learn any more on my own. The main problem I have is how to structure what it is I'm trying to improve upon. I also want more external perspective to help guide me towards becoming better in the web development field, but I don't feel like the company I'm at has developers with a modern web development skillset to offer that guidance. Unfortunately I work in an area where the web developer talent is pretty shallow. The general programmer talent pool is deep, but I still feel like the specialization towards webdev and modern practices just aren't here.
  • jgalar 9y ago
    IMO, it really depends on the context. If you are working with people who share your assessment of the current situation (both business and technical folks) and want to improve it, you'll have a great chance to learn from others' (and your) mistakes. However, constantly putting off fires, under the gun, in horrible code bases, is probably not a good way to learn how to design software... It's a good way to learn how to debug and reason about problems, which is also a valuable skill to develop, though.
  • user15672 9y ago
    I hate to say it, but I think the answer is "with difficulty". From my own experience, it's really hard to know what's bad, what's good and what's an acceptable workaround if you've never seen anything different. Myself, I got lucky and ended up working on a project after the start of my career with someone who could explain the whats and (more importantly) the whys of bad/good/ugly code bases. Generally, try and get some skill in being able to view a codebase from a high level. Draw it out on a whiteboard in boxes. Perhaps do this on other, pet projects first as it's nearly impossible to do this with a spaghetti-code project. If you can't pick out modular parts, then you have a big ball of mud. If you can, try and work on making and keeping them uncoupled. If you can, try and work on finding the natural boundaries of the other code you couldn't break up, and make those less coupled (you don't need to solve the coupling problems all at once!). Are there a mix of architectural patterns in the code? This is pretty common when you're working on a legacy project. It's what happens when you get someone who doesn't really know how to architect, or there were a bunch of folks throughout the history of the project who (probably) had the right intentions, but didn't get it finished. Or, and this is the worst, you had two or more team members trying to bend the project to their own preferences without communicating with each other. If this is the case, talk to your team, agree on one and then you can work towards getting the style consistant. You don't even need to pick the best one. Getting a project into a consistant state is better than having an ugly mix and match. Are there a bunch of mixed up design patterns floating around? Try and refactor those out as much as possible. Design patterns are great, and you should use them where appropriate. But if you find a lot of them nested within each other, it's not a good sign and probably indicates someone at some point swallowed a design pattern book and thought it would be a good idea to implement them. All of them. Nested patterns can more the likely be refactored out to simplify the code. Though again, make sure you understand what they are there for first. Otherwise you may be unpicking something intentionally complex that needs to exist to remove complexity elsewhere. What does the DB look like? Is it designed around the projects business logic? Is this sensible for your project? Personally, I dislike putting any business logic into the data storage layer but it might be sensible for your particular project, so YMMV. If business logic in the DB is causing nasty workarounds, then you may have something else to refactor there, though this may not be possible. Never refactor just for the sake of it! If you don't have buy-in for your ideas on how to improve a code-base from the rest of your team, you're going to be creating problems. You may also be missing critical information that your tech-lead knows about and made design decisions based on it. There have been several times I've tried to make things better as a Junior dev, only to find out I'd made some bad assumptions and created a mess. Don't refactor without tests either. The system may be reliant on strange code, so make passing tests before changing things. That way you at least know the behaviour hasn't changed.
  • rb808 9y ago
    I think its pretty common- and I think you're lucky. I was surprised to see the article say "It happens at least once in the lifetime of every programmer,". I think if you work on greenfield projects your whole career you're likely the one who's creating these 'steaming piles of manure'. By working on bad legacy projects you learn an awful lot of things about what works and what is a problem to maintain - it will make you a better developer. The only issue is if you always work on legacy stuff and never get to write greenfield you might get typecast as such. Whether that is a problem of not is up to you. Sounds like you care enough you can change when/if you want to.
    • couchand 9y ago
      I think you're setting up a false dichotomy. There are codebases other than just legacy and greenfield projects: high-quality, well-structured and well-maintained code. I would agree that if all you work on is greenfield you're probably making the messes others are cleaning up, but I don't think that means developers are bound to either make messes or clean them up. There are plenty of good, long-lived projects out there. Not every old project is legacy.
      • _virtu 9y ago
        This is what I've been wondering about. I don't care if the stack isn't the newest, or the tech is the shiniest. I'm just more interested in working on code that was _engineered_. That is code that was designed and then built. That's the problem I have with most of the code I'm working in. At my current place of work, we're not even using xmlhttprequest. We're using an antiquated xml library that's been hand rolled (xajax + major changes) to emulate our ajax requests. It's insanity to me that we're still in this mode.
    • remotehack 9y ago
      > I think if you work on greenfield projects your whole career you're likely the one who's creating these 'steaming piles of manure'. Eggzactly, well stated.
  • BurningFrog 9y ago
    I've learned an enormous amount from fixing terrible bugs in terrible code. One tip is that when you've finally found the actual line(s) with the bug, always try to understand why the programmer made that mistake. This has taught me much about what constructs are error prone.
    • _virtu 9y ago
      This is an interesting one. I'll have to think about this while I'm taking notes about bugs.
  • flukus 9y ago
    I'd never trust a developer that's only worked on green field projects, they're oblivious to the mess they leave because they aren't there long enough to feel the pain of their design decisions. So you've got one up on a lot of people there. Aside from your own projects, look for opportunities for other projects at work where you can start with a fresh technology stack. Some of these projects might be taking over the non-core functions of the main app. For instance, chances are a lot of the UI is sub-optimal (generic crud based) for some specific users. You might be able to create a slicker interface that makes it easier for them to do specific tasks that feed that data into the main database.
• taude 9y ago
  This is a good high-level overview of the process. I highly recommend that engineers working in the weeds, read "Working Effectively with Legacy Code" [1], as it has a ton of patterns in it that you can implement, and more detailed strategies on how to do some of the code changes hinted at in this article. [1] https://www.safaribooksonline.com/library/view/working-effectively-with/0131177052/ https://www.safaribooksonline.com/library/view/working-effec...
  • zimablue 9y ago
    Second this, this is one of the best coding books I've read. edit: it also gives a lot of similar advice to the article, big-bang rewrites often impossible, drawing a line somewhere in the application to do input-output diffing tests when you make a change
• bmh_ca 9y ago
  I mostly agree with this - bite-sized chunks is really the main ingredient to success with complex code base reformations. FWIW, if you want to have a look at a reasonably complex code base being broken up into maintainable modules of modernized code, I rewrote Knockout.js with a view to creating version 4.0 with modern tooling. It is now in alpha, maintained as a monorepo of ES6 packages at https://github.com/knockout/tko https://github.com/knockout/tko You can see the rough transition strategy here: https://github.com/knockout/tko/issues/1 https://github.com/knockout/tko/issues/1 In retrospect it would've been much faster to just rewrite Knockout from scratch. That said, we've kept almost all the unit tests, so there's a reasonable expectation of backwards compatibility with KO 3.x.
  • taude 9y ago
    Nice work on Knockout refactor. We still are actively using KO in our core product, and it's nice to see some legs left in the framework.
  • lloeki 9y ago
    > bite-sized chunks is really the main ingredient to success with complex code base reformations. An excellent talk about this is "The Scandalous Story of the Dreadful Code Written by the Best of Us" by Katrina Owen [0] [0] http://www.kytrinyx.com/talks/scandalous-story/ http://www.kytrinyx.com/talks/scandalous-story/
    • bmh_ca 9y ago
      That really is an excellent talk, thanks for sharing.
    • Anderkent 9y ago
      Is anyone else flabbergasted by the amount of effort required to mock a function call in Go, as described by this talk? Like, when at 3:20 the presenter says there's a thing you can do that makes it utterly trivial to test this feature, I immediately assumed she'll just have to write some mocks for the `comm` package, and plug that in. Cool, I guess she'll talk about a nice mocking library or something, or there's some business complexity involved where the comm package is particularly stateful and so difficult to mock. But no. The big difficulty seems to be that the language doesn't allow you to mock package-level functions; and so before you can mock anything you have to introduce an indirection - add an interface through which the notify package has to call things, move the code in the comm package into methods on that interface, correct all code to pass around this interface and call methods on it. Why would you choose to work in language that makes the most common testing action so painful?
• crankyadmin 9y ago
  Delete it... (Speaking from experience from work)
  • UXCODE 9y ago
    Is there someone who left the legacy code and became beneficial? As a result of my work experience, it was more beneficial to delete the legacy code and only provide the necessary functions when renewing the system.
• artursapek 9y ago
  Are there businesses building automation and tooling for working with legacy codebases? It seems like a really good "niche" for a startup. The target market grows faster every year :)
  • yeukhon 9y ago
    Specifically to do what?
    • artursapek 9y ago
      - Help developers build a high level understanding of the code and relationships between modules (with millions of lines, this is extremely hard) - Automate refactoring code to reduce complexity and cross-dependencies - Automate rewriting parts of code in mode modern languages and replacing it with some mediation layer (protobuf etc) I think industries like finance would welcome with open arms something that can do this. And it could go for a high price if it's still saving them money on countless hours of developer time. It's a growing cost every year to maintain legacy code that was written 3+ developer generations ago, and it's dangerous in cases where peoples' lives depend on the code being bug-free (infrastructure, medical) I started thinking about this problem a few days ago in a thread about AI https://news.ycombinator.com/item?id=14430652 https://news.ycombinator.com/item?id=14430652
      • yeukhon 9y ago
        I think some of this already exists using APM and code analysis. The only issue is existing toolsets often display ugly diagrams in like ER format or some flow diagram. Hard to read.
  • tjalfi 9y ago
    Semantic Designs[0] is one of several companies that sells software for working with legacy codebases and programming language translation. [1] is a SO post by one of their founders that describes some of the difficulties in programming language translation. [0] http://www.semdesigns.com/ http://www.semdesigns.com/ [1] https://stackoverflow.com/a/3460977/3465526 https://stackoverflow.com/a/3460977/3465526
• yeukhon 9y ago
  Healthcare.gov is a good example although not legacy codebase. Anyway, I think fixing small bugs and writing tests are best way to learn how to work with legacy system. This allows me to see what components are easier to rewrite/refactor/add more logging and instrumentation. Business cannot wait months before a bug is fixed just for the sake of making a better codebase. But I agree on database changes should be minimal to none as much as possible. Also, overcommunicate with your downstream customers of your legacy system. They may be using your interface in an unexpected manner. I have done a number of serious refactoring myself and god tests will save me a huge favor despite I have to bite teeth for a few days to a few weeks.
• btbuildem 9y ago
  Thanks for posting, some excellent high-level advice.
• quadcore 9y ago
  From what I've seen the most common mistake when starting working on a new codebase is to not read it all before doing any change. I really mean it, a whole lot of programmers simply dont read the codebase before starting a task. Guess the result, specially in terms of frustration.
  • rocky1138 9y ago
    Sometimes the code is so horribly written we have nothing else to try but to poke at it with a stick in different ways until it breaks.
• moonbug 9y ago
  This should be one of the first tasks that any aspiring career programmer has. It's an essential experience in making a professional.
• mfrisbie 9y ago
  Sometimes your inner desires to rewrite it from scratch can be overwhelming. https://alwaystrending.io/articles/software-engineer-entertains-erotic-fantasy-about-rewriting-entire-codebase-from-scratch https://alwaystrending.io/articles/software-engineer-enterta...
• mfrisbie 9y ago
  Sometimes your inner desires to rewrite it from scratch can be overwhelming. https://alwaystrending.io/articles/software-engineer-entertains-erotic-fantasy-about-rewriting-entire-codebase-from-scratch https://alwaystrending.io/articles/software-engineer-enterta...
• jefurii 9y ago
  Stick around that startup long enough and this a good set of things to do with your own code.
• lbill 9y ago
  I used to work on a messy legacy codebase. I managed to clean it, little by little, even though most of my colleagues and the management were a bit afraid of refactoring. It wasn't perfect but things kinda worked, and I had hope for this codebase. Then the upper management appointed a random guy to do a "Big Bang" refactor: it has been failing miserably (it is still going on, doing way more harm than good). Then it all started to go really bad... and I quit and found a better job!
• OutsmartDan 9y ago
  Big bang rewrites are needed in order to move forward faster. A huge issue with sticking to an old codebase for such a long time is that it gets older and older. You get new talent that doesn't want to manage it and leave, so you're stuck with the same old people that implemented the codebase in the first place. Sure they're smart, knowledgable people in the year 2000, but think of how fast technology changes. Change, adapt, or die.
  • jacquesm 9y ago
    A big bang rewrite will nine out of ten times slow you down, it will not accelerate things, and the most likely outcome is that not only will it be slower, it might fail entirely. It's a complete fallacy to think that you're going to do much better than the previous crew if you are not prepared to absorb the lessons they left behind in that old crusty code. So you'll have to learn them all over again. > Change, adapt, or die. Die it is then.
    • OutsmartDan 9y ago
      Software is only as good as the people that write it. In an ideal world, you'll have a team that specializes in this sort of things, can understand the business needs, and get it done. There are always risks with every action taken. You can't be scared to take a big risk for a bigger payout versus sucking it up and doing things the way they've been done for 15 years.
    • alkonaut 9y ago
      It's not a given that legacy code means "no people still around, no docs and no tests". I'm on a rewrite project and I'm 10 years in, and the whole crew from the last project (also around 15 years) is till in this project too. That helps. The causes of the big bang rewrite are usually not just "this code smells let's rewrite it" but rather that the old product reached some technical dead end. Perhaps it can't scale. Perhaps it's a desktop product written in an UI framework that doesn't support high DPI screens and suddenly all the customers have high DPI screens. Obviously in that situation you'd aim to just replace a layer of the application (a persistence layer, an UI layer) but as we all know that's not how it works. The cost of a rewrite shouldn't be underestimated - as you said there is no reason to believe that if it took 50 man years for the last team then the new team will take 50 too. But that is in itself not a reason to not do it.
• specialist 9y ago
  Sound advice. re: Write Your Tests I've never been successful with this. Sure, write (backfill) as many tests as you can. But the legacy stuff I've adopted / resurrected have been complete unknowns. My go-to strategy has been blackbox (comparison) testing. Capture as much input & output as I can. Then use automation to diff output. I wouldn't bother to write unit tests etc for code that is likely to be culled, replaced. re: Proxy I've recently started doing shadow testing, where the proxy is a T-split router, sending mirror traffic to both old and new. This can take the place of blackbox (comparison) testing. re: Build numbers First step to any project is to add build numbers. Semver is marketing, not engineering. Just enumerate every build attempt, successful or not. Then automate the builds, testing, deploys, etc. Build numbers can really help defect tracking, differential debugging. Every ticket gets fields for "found" "fixed" and "verified". Caveat: I don't know if my old school QA/test methods still apply in this new "agile" DevOps (aka "winging it") world.
  • mannykannot 9y ago
    >My go-to strategy has been blackbox (comparison) testing. Capture as much input & output as I can. Then use automation to diff output. Same here - you have an oracle, it would be a waste not to use it. You can probably also think of some test cases that are not likely to show up often in the live data, but I would contend that until you know the implementation thoroughly, you are more likely to find input that tests significant corner cases in the live data, rather than by analysis.
  • realcoopernurse 9y ago
    +1 for the split testing + diff approach. We've successfully used this several times to replace old components with new implementations.
  • douche 9y ago
    > My go-to strategy has been blackbox (comparison) testing. Capture as much input & output as I can. Then use automation to diff output. I wouldn't bother to write unit tests etc for code that is likely to be culled, replaced. I think that is precisely what the article advocates - although the definition of what end-to-end and integration tests are varies wildly from place to place. > First step to any project is to add build numbers. Semver is marketing, not engineering. Just enumerate every build attempt, successful or not. Then automate the builds, testing, deploys, etc. A thousand times this. And get to a point where the build process is reproducible, with all dependencies checked in (or if you trust your package manager to keep things around...). You should be able to pull down any commit and build it.
• weef 9y ago
  Great advice. Writing integration tests or unit tests around existing functionality is extremely important but unfortunately might not always be feasible given the time, budget, or complexity of the code base. I just completed a new feature for an existing and complex code base but was given the time to write an extensive set of end-to-end integration tests covering most scenarios before starting my coding. This proved invaluable once I started adding my features to give me confidence I wasn't breaking anything and helped find a few existing bugs no one had caught before!
  • UK-AL 9y ago
    I would argue that refactoring a legacy code base without tests, is not refactoring.
    • jacquesm 9y ago
      Russian Roulette?
  • humanrebar 9y ago
    > Writing integration tests or unit tests around existing functionality is extremely important but unfortunately might not always be feasible given the time, budget, or complexity of the code base. Bottom line: If the project cannot afford to properly maintain the code, it's a failure of the business model. Projects can be maintained indefinitely, but it costs money. And that means the project has to bring in enough money to pay for those maintenance costs. The options, as I see them: 1. Accept that this particular project, and those that intimately depend on it, has a lifecycle and will eventually die, either slowly or quickly. Prepare for that fact, staying ahead of the reaper by quitting, transferring to another project, etc. 2. Build a case to leadership that the project is underfunded long-term. This takes communication skills, persuasion skills, technical skills, and political skills. You'll need to go to all the stakeholders in their frame of reference and explain the risk involved in fundamentally depending on legacy code. Anyway, engineers tend to see the "legacy code" problem as a technical one. It is in the sense it takes technical work to fix it. But the root cause is a misallocation of resources. If the needed resources aren't there in the first place, the problem is a bad business model.
• macca321 9y ago
  Another thing you can do is start recording all requests that cause changes to the system in an event store (a la event sourcing). Once you have this in place, you can use the event stream to project a new read model (e.g.a new, coherent, database structure).
  • deleted 9y ago
    [deleted]
• deedubaya 9y ago
  Yeah, I've done this. It's frustrating and easy to burn out doing it because progress seems so arbitrary. Legacy upgrades are usually driven by large problems or the desire to add new features. Getting a grip on the code base while deflecting those desires can be hard. This type of situation is usually a red flag that the company's management doesn't understand the value of maintaining software until the absolutely have to. That, in itself, is an indicator of what they think of their employees.
  • jacquesm 9y ago
    > This type of situation is usually a red flag that the company's management doesn't understand the value of maintaining software until the absolutely have to. Recent conversation with the manager of a company: "I've yet to see anybody give me a good reason why we need to maintain the software we already built if it work." No kidding.
    • deedubaya 9y ago
      That's just a poor job of surfacing the consequences of not maintaining software by whoever built it.... unless their software is bug free... and we all know there is so much bug free software out there.
    • trnmura 9y ago
      Sadly, I think this is more of a rule than an exception...
• mannykannot 9y ago
  WRT architecture: In my experience, you would be lucky if you are free to change the higher level structure of the code without having to dive deeply into the low-level code. Usually, the low-level code is a tangle of pathological dependencies, and you can't do any architectural refactoring without diving in and rooting them out one at a time (I was pulling up ivy this weekend, so I was primed to make this comment!)
  • jacquesm 9y ago
    That's a good point I'll update the post. Thank you.
  • humanrebar 9y ago
    > ...you would be lucky if you are free to change the higher level structure of the code without having to dive deeply into the low-level code. The problem, in my mind, is that code can't be accurately modeled on one axis from "low level" to "high level". You can slice a system in many ways: - network traffic - database interactions - build time dependencies - run time dependencies - hardware dependencies - application level abstractions ...and certainly more. On top of that, the dimensions are not orthogonal. You might need to bump the major version of a library to support a new wire format, for example. Anyway, since there are many ways to slice a project, what is "high level" in on perspective can be "low level" from another. And vice versa.
• jlebrech 9y ago
  do refactoring you should have known at the time and not the brand new fangled way to do things, that way each new way fades into the other.
• apeace 9y ago
  > Do not fall into the trap of improving both the maintainability of the code or the platform it runs on at the same time as adding new features or fixing bugs. I don't disagree at all, but I think the more valuable advice would be to explain how this can be done at a typical company. In my experience, "feature freeze" is unacceptable to the business stakeholders, even if it only has to last for a few weeks. And for larger-sized codebases, it will usually be months. So the problem becomes explaining why you have to do the freeze, and you usually end up "compromising" and allowing only really important, high-priority changes to be made (i.e. all of them). I have found that focusing on bugs and performance is a good way to sell a "freeze". So you want feature X added to system Y? Well, system Y has had 20 bugs in the past 6 months, and logging in to that system takes 10+ seconds. So if we implement feature X we can predict it will be slow and full of bugs. What we should do is spend one month refactoring the parts of the system which will surround feature X, and then we can build the feature. In this way you avoid ever "freezing" anything. Instead you are explicitly elongating project estimates in order to account for refactoring. Refactor the parts around X, implement X. Refactor the parts around Z, implement Z. The only thing the stakeholders notice is that development pace slows down, which you told them would happen and explained the reason for. And frankly, if you can't point to bugs or performance issues, it's likely you don't need to be refactoring in the first place!
  • jacquesm 9y ago
    These problems tend to be systemic, not just tech problems and usually by the time we reach this stage management is a little more amendable to things like feature freezes than what the regular crew would be dealing with. There is a reason you get to that stage. So I can see how we have (much) more freedom when it comes to setting the time table and more diplomacy and better salesmanship might be required at an earlier stage. But then you can point to this comment here and suggest that it is probably much cheaper to do this in house than to hire a bunch of consultants to do it by the time the water is sloshing over the dikes.
    • gary__ 9y ago
      Extreme planned refactoring perhaps: "Many teams schedule refactoring as part of their planned work, using a mechanism such as "refactoring stories". Teams use these to fix larger areas on problematic code that need dedicated attention. Planned refactoring is a necessary element of most teams' approach - however it's also a sign that the team hasn't done enough refactoring using the other workflows." https://martinfowler.com/articles/workflowsOfRefactoring/ https://martinfowler.com/articles/workflowsOfRefactoring/
• korzun 9y ago
  > Before you make any changes at all write as many end-to-end and integration tests as you can. I don't agree with this. People can't write proper coverage for a code base that they 'fully understand'. You will most likely end up writing tests for very obvious things or low hanging fruits; the unknowns will still seep through at one point or another. Forget about refactoring code just to comply with your tests and breaking the rest of the architecture in the process. It will pass your 'test' but will fail in production. What you should be doing is: 1. Perform architecture discovery and documentation (helps you with remembering things). 2. Look over last N commits/deliverables to understand how things are integrating with each other. It's very helpful to know how code evolved over time. 3. Identify your roadmap and what sort of impact it will have on the legacy code. 4. Commit to the roadmap. Understand the scope of the impact for your anything you add/remove. Account for code, integrations, caching, database, and documentation. 5. Don't forget about things like jobs and anything that might be pulling data from your systems. Identifying what will be changing and adjusting your discovery to accommodate those changes as you go is a better approach from my point of view. By the time you reach the development phase that touches 5% of architecture, your knowledge of 95% of design will be useless, and in six months you will forget it anyways. You don't cut a tree with a knife to break a branch.
  • markatkinson 9y ago
    I gasped when I saw this article at the top of HN due to the relevance of it right now in my life. I am currently working on a real monolithic jambalaya that suffers from a lack of documentation, architecture, extreme abstraction, rampant tight coupling and no previous source control. Your point on performing architecture discovery and documentation is spot on. It has really helped me to strip away the mess and understand the flow of the logic and maybe even shine some light on the parts of code that are valuable.
    • korzun 9y ago
      > I gasped when I saw this article at the top of HN due to the relevance of it right now in my life. You are not the only one :)
    • 9y ago
• ransom1538 9y ago
  > Before you make any changes at all write as many end-to-end and integration tests as you can. ^ Yes and no. That might take forever and the company might be struggling with cash. I would instead consider adding a metrics dashboard. Basically - find the key points: payments sent, payments cleared, new user, returning user, store opened, etc. THIS isn't as good as a nice integration suite - but if a client is hard on cash and needs help - this can be setup in hours. With this setup - after adding/editing code you can calm investors/ceos'. Alternatively, if it's a larger corp it will be time strapped - then push for the same thing :)
  • dfabulich 9y ago
    I think instead of "as many as you can" it's "as many as you can afford."
• matt_s 9y ago
  Regarding instrumentation and logging - this can also be used to identify areas of the codebase that can possibly be retired. If it is a legacy application, there are likely areas that aren't used any longer. Don't focus on tests or anything in these areas and possibly deprecate them.
• Bahamut 9y ago
  Can't say I agree with the big bang rewrite part necessarily - at my last job, I found myself having to do significant refactors. The reason was that each view had its own concept of a model for interacting with various objects, which resulted in a lot of different bugs from one off implementations. My refactor had some near term pain of having to fix various regressions I created, but ultimately it led to much better long term maintenance.
• SideburnsOfDoom 9y ago
  > add instrumentation. Do this in a completely new database table, add a simple counter for every event that you can think of and add a single function to increment these counters based on the name of the event. The idea is a good one but the specific suggested implementation .. hasn't he heard of statsd or kibana?
  • jacquesm 9y ago
    Not available on all platforms. Think: mainframes, platforms no longer with the times, non-unix and so on. If you have access to a tool like that by all means use it, the specific implementation is not relevant, the article merely tries to show a simplest way to implement this very useful functionality that will work without limitation on just about anything that I can think of.
    • kod 9y ago
      Serious question - what platforms are you working on that you can't send a udp packet that looks like "somecounter:1|c"
      • jacquesm 9y ago
        Hehe. If you can't imagine that then you have a sheltered and probably very happy life. I don't care if it speaks ethernet, arcnet, twinax, X.25 or nothing at all, we'll find a way. By the time you can start sending UDP packets you are already on very solid footing. Be happy if your dev environment does not include an emulated version of the real hardware that mysteriously does not seem to be 100% representative of the real thing.
        • kod 9y ago
          Telling someone they lead a sheltered life isn't the same as actually answering the question. What actual systems have you worked on that were connected to a database, but couldn't send UDP?
• jofer 9y ago
  I agree with everything said, but I think they assumed a well-maintained and highly functionality legacy codebase. In my experience, there are a few steps before any of those. --- 1. Find out which functionality is still used and which functionality is critical Management will always say "all of it". The problem is that what they're aware of is usually the tip of the iceberg in terms of what functionality is supported. In most large legacy codebases, you'll have major sections of the application that have sat unused or disabled for a couple of decades. Find out what users and management actually think the application does and why they're looking to resurrect it. The key is to make sure you know what is business critical functionality vs "nice to have". That may happen to be the portions of the application that are currently deliberately disabled. Next, figure out who the users are. Are there any? Do you have any way to tell? If not, if it's an internal application, find someone who used it in the past. It's often illuminating to find out what people are actually using the application for. It may not be the application's original/primary purpose. --- 2. Is the project under version control? If not, get something in place before you change anything. This one is obvious, but you'd be surprised how often it comes up. Particularly at large, non-tech companies, it's common for developers to not use version control. I've inherited multi-million line code bases that did not use version control at all. I know of several others in the wild at big corporations. Hopefully you'll never run into these, but if we're talking about legacy systems, it's important to take a step back. One other note: If it's under any version control at all, resist the urge to change what it's under. CVS is rudimentary, but it's functional. SVN is a lot nicer than people think it is. Hold off on moving things to git/whatever just because you're more comfortable with it. Whatever history is there is valuable, and you invariably lose more than you think you will when migrating to a new version control system. (This isn't to say don't move, it's just to say put that off until you know the history of the codebase in more detail.) --- 3. Is there a clear build and deployment process? If not, set one up. Once again, hopefully this isn't an issue. I've seen large projects that did not have a unified build system, just a scattered mix of shell scripts and isolated makefiles. If there's no way to build the entire project, it's an immediate pain point. If that's the case, focus on the build system first, before touching the rest of the codebase. Even for a project which excellent processes in place, reviewing the build system in detail is not a bad way to start learning the overall architecture of the system. More commonly, deployment is a cumbersome process. Sometimes cumbersome deployment may be an organizational issue, and not something that has a technical solution. In that case, make sure you have a painless way to deploy to an isolated development environment of some sort. Make sure you can run things in a sandboxed environment. If there are organizational issues around deploying to a development setup, those are battles you need to fight immediately.
• maxxxxx 9y ago
  How do people handle this in dynamic languages like JavaScript? I have done a lot of incremental refactoring in C++ and C# and there the compiler usually helped to find problems. I am now working on a node.js app and I find it really hard to make any changes. Even typos when renaming a variable often go undetected unless you have perfect test coverage. This is not even a large code base and I find it already hard to manage. Maybe i have been using typed languages for a long time so my instincts don't apply to dynamic languages but I seriously wonder how one could maintain a large JavaScript codebase.
  • Thasc 9y ago
    It's not exactly a general solution fit for all situations and persons and purposes, but there's always TypeScript.
    • WorldMaker 9y ago
      Typescript's increasing ability to type check JS code without modification (especially if it already has JSDoc comments or is already using npm-installed libraries with type information) is moving it to be a better fit as a solution for more situations.
  • rocky1138 9y ago
    I've found integration testing to be very useful when dealing with JavaScript web stuff. If the desired output looks correct, you can usually work with the understanding that the JavaScript did its job. See Selenium. http://docs.seleniumhq.org/ http://docs.seleniumhq.org/
  • nulagrithom 9y ago
    Try TypeScript. Though I wouldn't think that test coverage needs "perfect" to catch a bad variable name, but maybe that's why there's so much obsessive tooling when it comes to coverage in the JavaScript world.
    • Roboprog 9y ago
      Don't forget that JS is often in a UI, doing asynchronous event/IO handling, so testing timing is important, not just spelling. (great, that's exactly the property names that object would have had, if it existed yet) That, and it's often reading in data (JSON or XML) from another system, and it is what it is, so see if it quacks or not. From the people that brought you SOAP, it's (drum roll) TYPE SCRIPT! It's not really solving my problems, just making more work.
• iamNumber4 9y ago
  Sometimes you get an entire septic tank full of... Code base that is non-existent, as the previous attempts were done with MS BI (SSIS) tools (for all things SSIS is not for) and/or SQL Stored procedures, with no consistency on coding style, documentation, over 200 hundred databases (sometimes 3 per process that only exist to house a handful of stored procedures), and a complete developer turn over rate of about every 2 years. with Senior leadership in the organization clueless to any technology. As you look at a ~6000 lines in a single stored procedure. You fight the urge to light the match, and give it some TLC ( Torch it, Level it, Cart it away) to start over with something new. Moral of the story, As you build, replace things stress to everyone to "Concentrate of getting it Right, instead of Getting it Done!" so you don't add to the steaming pile.
  • HelloNurse 9y ago
    Can you convince management that development in this situation is horrible and expensive and that there are better architectures?
• lol768 9y ago
  Any advice on what steps to take when the legacy codebase is incredibly difficult to test? I completely agree with the sentiment that scoping the existing functionality and writing a comprehensive test suite is important - but how should you proceed when the codebase is structured in such a way that it's almost impossible to test specific units in isolation, or when the system is hardcoded throughout to e.g. connect to a remote database? As far as I can see it'll take a lot of work to get the codebase into a state where you can start doing these tests, and surely there's a risk of breaking stuff in the process?
  • ef4 9y ago
    An after-the-fact test suite is a different beast than one written concurrently with the app. It's not worth trying to force one to be the other. Work from the outside in, keeping most of the system as a black box. Start with testing the highest-level behaviors that the business/users care about.
• kentt 9y ago
  > Do not ever even attempt a big-bang rewrite I'd love to hear a more balanced view on this. I think this idea is preached as the gospel when dealing with legacy systems. I absolutely understand that the big rewrite has many disadvantages. Surely there is a code base that has features such that a rewrite is better. I'm going to go against the common wisdom and wisdom I've practiced until now, and rewrite a program I maintain that is 1. Reasonably small (10k loc with a large parts duplicated or with minor variables changed). 2. Barely working. Most users cannot get the program working because of the numerous bugs. I often can't reproduce their bugs, because I get bugs even earlier in the process. 3. No test suite. 4. Plenty of very large security holes. 5. I can deprecate the old version. I've spent time refactoring this (maybe 50 hours) but that seems crazy because it's still a pile of crap and at 200 hours I don't think it look that different. I doubt it would take 150 hours for a full rewrite. Kindly welcoming dissenting opinions.
  • tyingq 9y ago
    There are cases where you can do a rewrite, but still avoid the big-bang cutover, by exposing the new app only to some subset of customers or transactions. That isn't possible with every app, of course. I think the gospel view is when you have to do both...rewrite and big bang cutover. Especially when there is no obvious fallback.
  • nawitus 9y ago
    I also disagreed with that part in the article. Big-bang rewrites can be just fine - but usually there are reasons it's not possible.
  • Yhippa 9y ago
    Not a dissenting opinion but I'd love to see some case studies on rewrites. As a consultant this is a frequent request and will probably be big business in the future as people migrate off of expensive legacy mainframe or other applications from the 80's, 90's, and possibly 2000's.
    • ef4 9y ago
      It's not "rewrite" that's bad, it's thinking you can cut over to a new system in a "big bang". Rewrites are definitely common and beneficial, but the successful ones always run the new code and the old code side-by-side for an extended period of time. Which means you're still tending and caring about the old code, even as you strive to direct most of your effort into the new code.
• sz4kerto 9y ago
  The OP has so many reasonable, smart-sounding advice that doesn't work in the real world. 1) "Do not fall into the trap of improving both the maintainability of the code or the platform it runs on at the same time as adding new features or fixing bugs." Thanks. However, in many situations this is simply not possible because the business is not there yet so you need to keep adding new features and fix bugs. And still, the code base has to be improved. Impossible? Almost, but we're paid for solving hard problems. 2) "Before you make any changes at all write as many end-to-end and integration tests as you can." Sounds cool, except in many cases you have no idea how the code is supposed to work. Writing tests for new features and bugfixes is a good advice (but that goes against other points the OP makes). 3) "A big-bang rewrite is the kind of project that is pretty much guaranteed to fail. No, it's not. Especially if you're rewriting parts of it at a time as separate modules My problem with the OP is really that it tells you how to improve a legacy codebase given no business and time pressure.
  • thehardsphere 9y ago
    >3) "A big-bang rewrite is the kind of project that is pretty much guaranteed to fail. >No, it's not. Especially if you're rewriting parts of it at a time as separate modules I guess it depends on what he considers to be a "big bang rewrite." I don't think any of the incremental approach you mention counts as one.
    • MaulingMonkey 9y ago
      My preferred definition here, is that a big bang rewrite is a monolithic rewrite so big it goes bang (fails), pushing things from "pretty much guaranteed to fail", to "is by definition a failure". You might end up rewriting the entire codebase through an incremental approach ala the Ship of Theseus through a series of smaller rewrites, but that's something very different and distinct from a "big bang rewrite" to me.
  • jacquesm 9y ago
    On the contrary, we do this work under extreme business and time pressure, sometimes existential pressure (as in: fail and the company fails). That's exactly why this list is set up the way it is: you will get results fast and they will be good results. If you want to play the 'I'm doing a sloppy job because I'm under pressure' card then consider this: the more pressure the less room there is for mistakes. Here is a much more play-by-play account of one of these jobs where management gave me permission to do a write-up as part of the deal: https://jacquesmattheij.com/saving-a-project-and-a-company https://jacquesmattheij.com/saving-a-project-and-a-company (For obvious reasons management usually does not give such permission, nobody wants to admit they let it get that far on their watch, I did my best to obscure which company this is about.)
• cessor 9y ago
  I'd add a prerequisite to the top of this list: - Get a local build running first. Often, a complete local build is not possible. There are tons of dependencies, such as databases, websites, services, etc. and every developer has a part of it on their machine. Releases are hard to do. I once worked for a telco company in the UK where the deployment of the system looked like this: (Context: Java Portal Development) One dev would open a zip file and pack all the .class files he had generated into it, and email it to his colleague, who would then do the same. The last person in the chain would rename the file to .jar and then upload it to the server. Obviously, this process was error prone and deployments happened rarely. I would argue that getting everything to build on a central system (some sort of CI) is usefull as well, but before changing, testing, db freezing, or anything else is possible, you should try to have everything you need on each developer's machine. This might be obvious to some, but I have seen this ignored every once in a while. When you can't even build the system locally, freezing anything, testing anything, or changing anything will be a tedious and error prone process...
  • jacquesm 9y ago
    Added, thank you. Also added a bit about the very obvious backup that you need to make before starting any work at all. Just in case...
  • flukus 9y ago
    > I would argue that getting everything to build on a central system (some sort of CI) is usefull as well, but before changing, testing, db freezing, or anything else is possible, you should try to have everything you need on each developer's machine. I'd extend this and say that the CI server should be very naive as well. It's only job is to pull in source code and execute the same script (makefile, whatever) that the developers do. Maybe with different configuration options or permissions, but the developers should be able to do everything the CI server does in theory. A big anti pattern I see is build steps that can only be done by the CI server and/or relying on features of the CI server software.
• pinaceae 9y ago
  First and foremost, do not assume that everyone who ever worked on the code before is a bumbling idiot. assume the opposite. If it's code that has been running successfully in production for years, be humble. Bugifxes, shortcuts, restraints - all are real life and prevent perfect code and documentation under pressure. The team at Salesforce.com is doing a massive re-platforming right now with their switch to Lightning. Should provide a few good stories, switching over millions of paying users, not fucking up billions in revenue.
• pc86 9y ago
  I've been a part of several successful big-bang rewrites, and several unsuccessful ones, and saying that if you're smart they're not on the table is just flat out wrong. The key is an engaged business unit, clear requirements, and time on the schedule. Obviously if one or more of these things sounds ridiculous then the odds of success are greatly diminished. It is much easier if you can launch on the new platform a copy of the current system, not a copy + enhancements, but I've been on successful projects where we launched with new functionality.
  • jacquesm 9y ago
    I've yet to see a large system with lots of subsystems rewritten in one go, but I'm more than open to being convinced that it can be done so if you could please do a write-up of how such a project was managed. The ones I have seen - and this is actually one of the major reasons the clean-up crew gets called in the first place - is big bang rewrite projects gone astray. One huge problem with rewrites of old code is that the requirements are no longer known or even misunderstood.
    • alkonaut 9y ago
      The biggest problem with "the new system" is that it's rarely a rewrite of the second system. Obviously someone liked the old system otherwise it wouldn't be rewritten. But the business case for the new system isn't just lower maintenance cost, higher performance, a modern look etc. It's always going to be all those new features. That's what sinks the new project.
• hinkley 9y ago
  It's my turn to disagree with something in the article. > Before you make any changes at all write as many end-to-end and integration tests as you can. I'm beginning to see this as a failure mode in and of itself. Once you give people E2E tests it's the only kind of tests they want to write. It takes about 18 months for the wheels to fall off so it can look like a successful strategy. What they need to do is learn to write unit tests, but You have to break the code up into little chunks. It doesn't match their aesthetic sense and so it feels juvenile and contrived. The ego kicks in and you think you're smart enough you don't have to eat your proverbial vegetables. The other problem is e2e tests are slow, they're flaky, and nobody wants to think about how much they cost in the long run because it's too painful to look at. How often have you see two people huddled over a broken E2E test? Multiply the cost of rework by 2.
• d--b 9y ago
  I agree with most of this, though I think it doesn't dive into the main problem: Freezing a whole system is practically impossible. What you usually get is a "piecewise" freeze. As in: you get to have a small portion of the system to not change for a given period. The real challenge is: how can you split your project in pieces of functionalities that are reasonably sized and replaceable independently from each other. There is definitely no silver bullet for how to do this.
  • jacquesm 9y ago
    I could probably do a better job of making that clear in the article. The whole point is to iterate and to lock and release parts selectively so you are never working on more than one thing at the time.
• kevan 9y ago
  >Use proxies to your advantage At my last gig we used this exact strategy to replace a large ecommerce site piece by piece. Being able to slowly replace small pieces and AB test every change was great. We were able to sort out all of the "started as a bug, is now a feature" issues with low risk to overall sales.
• rattray 9y ago
  > Yes, but all this will take too much time! I'm actually quite curious; how long does this process typically take you? What are the most relevant factors on which it scales? Messiness of existing code? Number of modules/LOC? Existing test coverage?
  • jacquesm 9y ago
    Good questions. How long it takes depends on the mandate given by management. Sometimes it's 30 days to get from zero to something stable and incrementally improvable at which point we hand back to the company with maybe a transition period where we still manage the project. Sometimes it is just a feasibility study in which case it can be even shorter. But if it is boots-in-the-mud (which is where the real money is) then it can be up to a year. It scales just fine provided you have the people and this is more often than not a huge problem. It's happened that we had to leave people in place for months or even years after the project was in essence done simply because as soon as our backs were turned it was back to the usual methods. That's actually really frustrating when it happens. Existing test coverage can speed things up but if the tests are brittle or otherwise not helpful can actually make things much worse. As for number of modules or LOC: if you're doing a platform switch that can really eat up time, if it is just to bring things under control then it does not really matter much. One you did not mention, but which can greatly impact the speed with which you can move is the quality of existing documentation. If there is anything at all, especially up to date requirements documentation that can serve as a tie breaker between a suspected bug or a feature it can make a huge difference.
    • rattray 9y ago
      Very interesting, thanks!
• alexwebb2 9y ago
  > How to Improve a Legacy Codebase When You Have Full Control Over the Project, Infinite Time and Money, and Top-Tier Developers edit: I'm being a little snarky here, but the assumptions here are just too much. This is all best-case scenario stuff that doesn't translate very well to the vast majority of situations it's ostensibly aimed at.
• jhgjklj 9y ago
  The biggest problem in improving legacy codebase is that the people who have involved with have been too long and are completely using old techinques and as a new developer you can not change them, they will change you which means its hard to improve.
• alexeiz 9y ago
  I was in this situation more than once. My actions are usually these: * Fix the build system, automate build process and produce regular builds that get deployed to production. It's incredible that some people still don't understand the value of the repeatable, reliable build. In one project, in order to build the system you had to know which makefiles to patch and disable the parts of the project which were broken at that particular time. And then they deployed it and didn't touch it for months. Next time you needed to build/deploy it was impossible to know what's changed or if you even built the same thing. * Fix all warnings. Usually there are thousands of them, and they get ignored because "hey, the code builds, what else do you want." The warning fixing step allows to see how fucked up some of the code is. * Start writing unit tests for things you change, fix or document. Fix existing tests (as they are usually unmaintained and broken). * Fix the VCS and enforce sensible review process and history maintenance. Otherwise nobody has a way of knowing what changed, when and why. Actually, not even all parts of the project may be in the VCS. The code, configs, scripts can be lying around on individual dev machines, which is impossible to find without the repeatable build process. Also, there are usually a bunch of branches with various degrees of staleness which were used to deploy code to production. The codebase may have diverged significantly. It needs to be merged back into the mainline and the development process needs to be enforced that prevents this from happening in the future. Worst of all is that in the end very few people would appreciate this work. But at least I get to keep my sanity.
  • mattmanser 9y ago
    I've always found it remarkably quick to fix warnings too, tends to be the same mistakes over and over.
• safek 9y ago
  > Do not ever even attempt a big-bang rewrite Really? Are there no circumstances under which this would be appropriate? It seems to me this makes assumptions about the baseline quality of the existing codebase. Surely sometimes buying a new car makes more sense than trying to fix up an old one?
  • kyberias 9y ago
    Your car buying analogy is flawed. When you buy a new car, someone has built it for you. It's cost effective because the manufacturer builds a great number of them. You can be fairly certain that it works and if it doesn't you'll have a guarantee. When you rewrite a software system, you do it yourself. You don't know whether you'll succeed. You might end up with worse end-results. The assumption here is that no off-the-shelf software can be used to replace it. Hence rewrite.
    • tim333 9y ago
      Also see Splosky's well known essay https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
  • sbov 9y ago
    For what the OP is talking about, I would say to never attempt a rewrite. The only caveat is if you have spent the time to truly understand the codebase, then maybe you can do it. Most people advocate a rewrite because they don't WANT to understand the codebase. Even if you understand the codebase, it's pretty dangerous, but at least you have some idea of what you're saying you will rewrite. So yeah, it can happen, but if you are in the situation that you have the knowledge and experience to override that rule, then you have the knowledge and experience to know that you CAN override that rule. It sounds a little circular, but it's how I tend to aim my broadly-given advice. If someone knows what they're doing, they should be able to recognize when they can ignore your advice. Anything else would have to be tailored to each specific instance, which isn't plausible in a blog post.
• busterarm 9y ago
  All of this seems to focus on the code, after glossing over the career management implications in the first paragraph. I've done this sort of work quite a number of times and I've made mistakes and learned what works there. It's actually the most difficult part to navigate successfully. If you already have management's trust (i.e., you have the political power in your organization to push a deadline or halt work), you're golden and all of the things mentioned in the OP are achievable. If not, you're going to have to make huge compromises. Front-load high-visibility deliverables and make sure they get done. Prove that it's possible. Scenario 1) I came in as a sub-contractor to help spread the workload (from 2 to 3) building out a very early-stage application for dealing with medical records. I came in and saw the codebase was an absolute wretched mess. DB schema full of junk, wide tables, broken and leaking API routes. I spent the first two weeks just bulletproofing the whole application backend and whipping it into shape before adding new features for a little while and being fired shortly afterwards. Lesson: Someone else was paying the bills and there wasn't enough visibility/show-off factor for the work I was doing so they couldn't justify continuing to pay me. It doesn't really matter that they couldn't add new features until I fixed things. It only matters that the client couldn't visibly see the work I did. Scenario 2) I was hired on as a web developer to a company and it immediately came to my attention that a huge, business-critical ETL project was very behind schedule. The development component had a due date three weeks preceding my start date and they didn't have anyone working on it. I asked to take that on, worked like a dog on it and knocked it out of the park. The first three months of my work there immediately saved the company about a half-million dollars. Overall we launched on time and I became point person in the organization for anything related to its data. Lesson: Come in and kick ass right away and you'll earn a ton of trust in your organization to do the right things the right way.
• logicallee 9y ago
  This says, near the end, "Do not ever even attempt a big-bang rewrite", but aren't a LOT of legacy in-house projects completely blown out of the water by well-maintained libraries of popular, modern languages, that already exist? (In some cases these might be commercial solutions, but for which a business case could be made.) I'm loath to give examples so as not to constrain your thinking, but, for example, imagine a bunch of hairy Perl had been built to crawl web sites as part of whatever they're doing, and it just so happens that these days curl or wget do more, and better, and less buggy, than everything they had built. (think of your own examples here, anything from machine vision to algabreic computation, whatever you want.) In fact isn't this the case for lots and lots of domains? For this reason I'm kind of surprised why the "big bang rewrite" is, written off so easily.
• user5994461 9y ago
  Agreed about the pre-requirements: Adding some tests, reproducible builds, logs, basic instrumentations. Highly disagree about the order of coding. That guy wants to change the platform, redo the architecture, refactor everything, before he starts to fix bugs. That's a recipe for disaster. It's not possible to refactor anything while you have no clue about the system. You will change things you don't understand, only to break the features and add new bugs. You should start by fixing bugs. With a preference toward long standing simple issues, like "adding a validation on that form, so the app doesn't crash when the user gives a name instead of a number". See with users for a history of simple issues. That delivers immediate value. This will give you credit quickly toward the stakeholders and the users. You learn the internals doing, before you can attempt any refactoring.
• stephenwilcock 9y ago
  It is great to see more people sharing their strategies for managing legacy codebases. However, I thought it might be worth commenting on the suggestion about incrementing database counters: > "add a single function to increment these counters based on the name of the event" While the sentiment is a good one, I would warn against introducing counters in the database like this and incrementing them on every execution of a function. If transactions volumes are high, then depending on the locking strategy in your database, this could lead to blocking and locking. Operations that could previously execute in parallel independently now have to compete for a write lock on this shared counter, which could slow down throughput. In the worst case, if there are scenarios where two counters can be incremented inside different transactions, but in different sequences (not inconceivable in a legacy code), then you could introduce deadlocks. Adding database writes to a legacy codebase is not without risk. If volumes are low you might get away with it for a long time, but a better strategy would probably just to log the events to a file and aggregate them when you need them.
• jscn 9y ago
  Genuinely would like to know how anyone has managed to do both of: > write as many end-to-end and integration tests as you can and > make sure your tests run fast enough to run the full set of tests after every commit