9 ms·
Lessons learned from rewriting code
- jasonpeacock 7y ago> our CTO was handling everything about AntiMalware. He was the main developer... There's (one) of your problems. Your CTO (just like managers) should not be coding, they should be defining and leading the technical strategy of the company.
- reallydude 7y ago> they should be defining and leading the technical strategy of the company. At most companies (small to medium and a large number of industries), that isn't a fulltime job. Most companies have IT as support or are a very small segment of the total business.
- wolco 7y agoIf you happen to be part of a two person team without any money you're CTO is going to need to code. At a team of 4/5 the handoff begins by employee 11 they shouldn't be coding perhaps firefighting if necessary.
- jasonpeacock 7y agoAs part of a two person team, there are no CEO/COO/CTO/CFO roles...you're just two people building something. It's a joke to give yourself a management title when there's nothing to manage.
- Juliate 7y agoIt's not a management title, it's a role. Being a 2 people company or a 20k+, you still have to manage financial, technical, operation aspects of your business. Same role name, different implications depending on scale.
- ska 7y agoIt’s a role (CTO), but not one that exist on a small team. If you are a small handful of people starting off, you don’t need a CFO, you need a bookkeeper and an accountant. You don’t need a CTO, you need a tech lead. All that rest is a mix of confusion and puffery.
- Juliate 7y agoThe puffery is in the inflation of the supposed prestige of such titles (both in puny and large settings).
- jasonpeacock 7y agoThere's a level of strategy, planning, influence, and execution that is completely missing at the small level - it's not even a matter of scale. Ignoring that when giving yourself a C-level title is what I take issue with.
- Juliate 7y agoYou're saying exactly the same thing I'm saying. Same role name, different situational implications. Being a mayor of a small village or of Paris or New-York, you're still the mayor.
- brightball 7y agoSeeing CEO on the business cards of single person companies always makes me chuckle.
- bdcravens 7y agoDepending on what form your business takes and where you file, you may have to designate who the CEO or President is. Of course, you needn't always put that on a business card.
- deleted 7y ago[deleted]
- usrusr 7y agoIn a two person company you don't have a CTO and a CEO, you have "The Code Guy/Gal" and "The Other One".
- cosmodisk 7y agoAll these CFO,CTO,CEO,CMO and etc. titles make me laugh when it comes to small companies. These titles make sense when we talk about the likes of Google,MS, Facebook. If it's a shop of 10 people, you are not a CTO,you are a tech lead at best with a few less experienced developers.The 2 co-owners of the business I work for call themselves joint CEOs, with only one line of management separating them from the lowest level employees.I could call myself Vice President or COO or something like this but that'd be idiotic knowing that I only manage a team of 10 people.This obsession and vanity with titles is complete nonsense.
- mebassett 7y agofancy titles are useful when you are dealing with Big Enterprise Clients. "CTO of TinyStartup" means you get taken slightly more seriously than "some rando building stuff". In fact, it's not even with the people you're directly dealing with where this becomes useful (as, if you're selling to Big Enterprise Client, you likely have a network inside the company anyhow). It's when the person you have a relationship with has to convince his/her boss to approve an invoice from your little shop.
- bdavis__ 7y agoWould you rather buy $100K of software from a "Senior Sales Representative" or a "VP of Sales, Western US region". Titles mean a lot to certain people, even if they are all puffery.
- cosmodisk 7y agoPersonally I wouldn't care. The reason is, that regardless of the company I'd go on LinkedIn just to see who the person is. The deciding criteria would not be the sales rep's title.The functionality of the product, support, chances of them staying alive for the next few years and etc. are much more important. However, I appreciate that's not necessarily the case of how a lot of people think.
- cosmodisk 7y agoHave to agree with this. Ever since I got promoted, the way people( outside the company) tend to talk to me has changed a lot.Even though good part of my daily activities are the same, people tend to take me more seriously.It does help to chase people to make sure they do things or even for the customers,who want to talk to 'the manager', even though I'd help as much as any other colleague of mine...
- Numberwang 7y agoThis makes me think of groovehq who just recently forced all their customers on to their new rewritten from scratch product. This after one full year of their customers begging them not to. So now they have a new modern (in the 2017 sense) product which is worse in many ways and with a whole set of new bugs. And some very unhappy customers. Improve your products. Don't replace them.
- kostarelo 7y agoWow, I kinda work in a legacy (3-4 years old), not very easy codebase, where new releases affect other parts and bring new bugs, but I cannot imagine the state of yours. Not being able to do an update?
- joefourier 7y agoHow can a 3-4 year old codebase be legacy? That's not a lot of time for the platform or technologies it uses to become crusty and obsolete.
- defined 7y agoSome codebases are legacy from the word go. Legacy is more about the maintainability than the age of the codebase.
- randallsquared 7y agoMaybe it wasn't built with containerization in mind. ;)
- bdcravens 7y agoI often surprise myself - I see commits that are 4 years old that I could have sworn I implemented just a few months ago
- notdang 7y agomaybe it's javascript
- davidjnelson 7y agoIn the book Working Effectively With Legacy Code the author defines “legacy” as “code without tests”.
- joefourier 7y agoThat's not really a popular or even useful definition. According to that author, a Go microservice I wrote yesterday would be legacy code, while a 25 year old Turbo Pascal scientific instrumentation program running on MS-DOS, which some poor soul still has to maintain is not, because the programmers wrote a few tests for it back in the day?
- devxpy 7y agoI never really got the reason for doing this var condn = ...; if (condn) { return true; } return false;
- deleted 7y ago[deleted]
- bberrry 7y agoBecause it gives you the opportunity to descriptively name the expression, sparing others from parsing it mentally.
- Liron 7y agoDo you also think `if ((x > 3) == true)` is more descriptive than `if (x > 3)`?
- sombremesa 7y agoCan't the returning function be named appropriately?
- bberrry 7y ago
- Liron 7y agoHe lost me at the very beginning due to clumsy handling of boolean values.
- Liron 7y agoJoel Spolsky wrote a classic piece about Netscape's rewrite in 2000 [1], the article links to it too. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- FascistDonut 7y agoThis is referenced in the article.
- kasperni 7y ago"Rewriting a system from the ground up is essentially an admission of failure as a designer. It is making the statement, “We failed to design a maintainable system and so must start over.” " Or maybe the domain you are working with was vastly more complex than you imagined, so your nice little system turned out to be more of a prototype than a finished product. Or maybe the features needed was vastly different from what you expected when you started implementing the thing. I'm sure everyone can come up with a million other reasons. But the problem I have every time with these "Never rewrite your software" stories. Is that they are almost always one-sided. You rarely hear about the stories where a company decided not to do a rewrite and was out-innovated because every little feature took months to implement. Or failed because they could not attract competent developers because their codebase was a complete mess. Yes, the rewrite might have been a partial or complete failure. But would the patient have survived without the operation anyway?
- nradov 7y agoOr maybe your customer base and system load has expanded by an order of magnitude beyond the original design.
- save_ferris 7y agoWriting scalable software is all about being as flexible as possible. If system load expanded by an order of magnitude and the original design wasn't flexible enough to scale with the load, that's a bad software design.
- wildrhythms 7y agoSure but then risk being dinged by the customer/management for "over-engineering something simple." How do you determine where the cut off point is?
- TuringNYC 7y agoOr perhaps the first project was budgeted to only handle limited scale and prove feasibility. In which case it was a success, and now that feasibility is proven, scale can be budgeted for. I mean, if people tried to build for mega scale from day one, every pie in the sky idea I have would start with Spark clusters and load balancers and Kubernetes...which obviously is all expensive, time consuming, and may simply not be worth the cost given the business objective.
- GoToRO 7y agoI can not comment unless I see the code(s).
- ChristianGeek 7y agoAnd yet you did.
- GoToRO 7y agoDid I really?
- cottonseed 7y agoWhen I hear about rewrites, I always think about this comment: https://news.ycombinator.com/item?id=11554288 https://news.ycombinator.com/item?id=11554288
- itronitron 7y agoSo true, the opportunity to leave things better than as you found them is one of the primary rewards in fixing bugs in existing/old/legacy systems.
- nemothekid 7y agoA pattern I've started seeing with most "bad" rewrites is that "bad" rewrites are probably usually motivated by burnout than actual architectural problems. People get tired of working on the code, and they would rather be working on something else hence "let's rewrite". When you hear about "good" rewrites it's almost never framed in a way that is "the old code is bad and complicated." It's usually driven by an architectural need - for example it started in Python, but now we have scaled up and in order to avoid $1MM/month a rewrite in Go/Rust/C would be beneficial. Or, in the case of changing marketing needs, it can be "we now better understand our customers and a rewrite can help us provide a more stable interface and help us iterate faster". However in this case a full 100% rewrite is never the case and it's usually done piecemeal or by the new system simply proxying requests to the old system. When the excuse is simply the "code is a mess", then not only are you admitting that the original team failed to design a maintainable system, you are also trusting them to not make the same mistake again. And if their whole motivation is "the code is a mess", they will probably fuck up the redesign, as messy code is a symptom of bad design but not the root cause. If the root cause is burnout, and that isn't addressed, the rewrite may be just as bad.
- rightbyte 7y agoIterational design is a thing. If the same team that did the crappy code gets a chance to learn from the mistakes and rewrite it part by part now when they have the whole picture, it's easier and the end result might be mantainable and nice.
- marcinzm 7y agoI've heard that the worst code is the 2nd version. You try to fix everything from the 1st version (real or perceived) and over correct as a result.
- bdavis__ 7y agoHowever the third iteration is something that you can be proud of. (but in my experience, 1/2 through the second iteration the funding gets cut off)
- skybrian 7y agoAlso consider how much time you'd lose by not forking a well-designed, open source codebase.
- apo 7y ago> Consider refactoring before taking a step to code rewriting: Refactoring is only an option for anything but a trivial code base if a suite of automated tests have already been written. The article makes no mention of tests. I suspect the reason is that there were none. Refactoring would most likely have been impossible until an automated test suite had been written. Many developers are confused about the purpose of an automated test suite. It's not to make your designs better. It's to allow you to refactor the spaghetti that will inevitably arise.
- Ma8ee 7y agoThat is just not true. There’s always the option of refactoring, make the necessary changes and manually test the system. Even though I’m a big fan of automated testing (not only unit tests), most codebases don’t have complete test coverage, but we still refactor and change the systems.
- hpolatyuruk 7y agoThe article is not about refactoring. It's about rewriting the software. So that's why I didn't mention about refactoring in details. Automated testing is part of the refactoring. You will change a small amount of code then run tests to see it works as expected. This is how refactoring works. No need to mention abou that.
- kstenerud 7y agoI've been involved in 3 separate rewrites in different industries, at companies of different sizes, over the course of 20+ years. Not one of them went well. I've also been involved in maintaining legacy code during that time. The code sucked, and changes were slow, but it went a LOT better than the rewrites did. We actually shipped things.
- yibg 7y agoThe problem is it’s not linear. You can ship until you can’t (or can but very very slowly). It’s taking all of the hit now or a little at a time until things completely grind to a haunt. I’ve been involved in a few rewrites as well. Some went well and we had a noticeable productivity gain. Some went badly and we were stuck for a long time fixing bugs and corner cases we had already addressed in the legacy stack. It really depends on the situation.
- gchamonlive 7y agowhat motivated those rewrites?
- kstenerud 7y agoOne was pure vanity. We replaced a perfectly functional codebase in order to support a bunch of new functionality more easily. In the end, what we actually did implement could have easily been done on the old architecture. Another one was a 15 year old codebase. We were also switching to Java. The process had taken 3 years by the time I arrived, and it was eventually canceled. They'd brought in a bunch of consultants (I was one of them), and everyone had their own idea and their own architecture for their parts. The pieces didn't play well together. The 3rd one was just your standard rewrite of a spaghetti codebase. I still think it was the right choice to rewrite, but it took too long and the project was canceled.
- gchamonlive 7y agoI see... Rewrites have to be both justifiable and shouldn't take too long. I am telling that to myself, we have an old code base in php that we would love to rewrite in python, but it must be justifiable. I believe we will do it incrementally, with each new function written in the python framework we have and any code maintenance that won't take too long to rewrite
- armitron 7y ago10 years as a developer and the only projects of yours I see on GitHub - 2 in total - are trivial and contain elementary errors and anti-patterns. You chose to link to your GitHub profile thus any critique is fair in my view, especially when you put forth your "10 years" of experience. Your blog is also full of the typical low-SnR, 0-substance, platitude-ridden posts one encounters when dealing with people who are terrible engineers and are winging it / trying to appear clued in / craft a persona. I am sorry but I can’t take anything you write seriously.
- dang 7y agoPersonal attacks will get you banned here. Maybe you don't owe the person you're attacking better, but you owe the community better. Would you mind reviewing the site guidelines and taking the spirit of this site more to heart when posting here? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html You might also find these links helpful for getting the intended idea: https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/hackernews.html https://news.ycombinator.com/hackernews.html http://www.paulgraham.com/trolls.html http://www.paulgraham.com/trolls.html http://www.paulgraham.com/hackernews.html http://www.paulgraham.com/hackernews.html
- curtis 7y agoAs a general rule you shouldn't rewrite everything from scratch. This does not mean that you shouldn't, on occasion, rewrite some parts of your system. Rewrites are expensive. On the other hand some code you think is bad really is bad, and it will be a net win to rewrite it. To make a call like this, you really do need a pretty good understanding of your codebase, your business, your testing exposure, the capabilities of your team, and probably some other stuff as well. In reality, your understanding is never going to be as good as you really need it. This doesn't mean you should never rewrite any portion of your codebase, it just means that you really need to reduce the scope a lot to reduce the risk to a manageable amount.
- giancarlostoro 7y agoAt my last job all I ever did was rewrite software from scratch. Sometimes the technical debt has led you beyond a point of being able to continue using the solution you have. For example Flash (soon to be ditched by mainstream browsers!) and Silverlight (no longer supported on modern browsers), and heck Java on frontend as well. I rewrote an entire Silverlight application to HTML5 and JS for the frontend and the backend with ASP .NET Core. Then I also rewrote a Java servlet that I never directly touched the codebase for but I was told you needed a special VM just to get it to compile, I rewrote it in Python using CherryPy. Sometimes the technical debt is not worth keeping around. I sure could of modernized the Java but Pythons a lot more approachable to the rest of the team and anybody can understand what is going on at a simple glance. I can run Python on Ubuntu without a hassle since its out of the box (our main OS in that office).
- gfodor 7y agoTurning one piece of software into another is something which offers a spectrum of techniques which range from micro-refactors up to full rewrites. If you find yourself reaching for the full rewrite option, you should realize you have suddenly reached to the very far, far end of the spectrum of options and are basically taking an extreme viewpoint. If you haven't gamed out and really tried to rule out options across the entire spectrum, you've failed -- not just because you've been negligent in terms of planning and trying to make the best choice for your organization/team, but also because if you do end up doing the rewrite, you've made a specific choice that, unlike most others, infuses a large amount of existential risk to the organization and product. In the case of a full re-write, there really must be no other way to accomplish the goal. How many rewrites happen when there was literally no other way? Engineers seem to have a bottomless well of creativity when it comes to solving problems, but that creativity often evaporates when focused on the problem specifically of "How the hell do we get from A to B without a rewrite?" No pain, no gain.
- garganzol 7y agoI'm pretty sure I know the exact product being described. It is Anti-Malware by Malwarebytes. Being a customer for many years, I saw the downward trend and it corresponds to that described in the article, both chronologically and by meaning. P.S. I can add up a lot to this story from a customer's point of view. Let me know if you are interested, I will continue in a comment below.
- rwallace 7y agoYes please!
- novok 7y agoI've been through successful rewrites. I think rewrites make sense when your product has scaled very fast with 50x more engineers or 50x more demands on it, so the context in what was made doesn't make sense anymore. You need a faster language, a stricter architecture and code discipline to deal with +100 engineers working on one product vs the previous 3, etc. Rewrites in that case is more like your old wooden 4 story office is too small, so now you need to build a skyscraper out of steel and concrete. While the steel concrete structure is being built, you still keep an maintain your old office since it's running your business, and you have the budget to simultaneously maintain both. Steel structures require different building techniques than your wooden one. That kind of hyper scaling is really rare outside of SF venture capital companies, so in most cases, you shouldn't do rewrites.
- abraae 7y agoIt's also possible that things that grow 50x tend to be newer, meaning less gradual accretion of cruft in the code. Being newer also means more remaining institutional knowledge in the organization about the technology. For 50x growth, the new thing we are thinking of rewriting probably also has some simple core purpose (like some kind of social network), making it easier to grok the existing code and indeed to rewrite it. By contrast a decrepit old payroll system will not have the growth dynamic, and will not be at all simple to rewrite - the code will be packed with small fixes and strange little features that are actually critically important, and that it would be disastrous to omit in a rewrite. These tie in with your last point.
- jcoffland 7y ago> 1. Are you ready to throw away all that knowledge? This is a massive fallacy. Rewriting code does not have to throw away knowledge. The old code is a specification for the new. You just have to read it. I've been involved in several successful rewrites and I always use the old code as a precise description of how the software is supposed to work. You have to take the time to understand it. And if you don't understand the old code, you really shouldn't be rewriting it.
- deleted 7y ago[deleted]
- AnimalMuppet 7y agoThe code as "precise" description: The problem is that you can't tell if that weird "if" is correctly handling a corner case, or incorrectly handling a corner case, or is now a "can't happen" left behind after something else changed. You can't tell which from the code. That means that tribal knowledge is crucial for a successful rewrite. And that means that you need to do the rewrite while you still have that knowledge available.
- jcoffland 7y agoI don't agree. The code may not do precisely what was intended but it is deterministic. If you understand the language, you can understand the code.
- AnimalMuppet 7y agoYes, you understand what it says. You don't know what it should say, though. The code can't tell you that. [Edit: "Better architecture, but bug-for-bug compatible" probably isn't the goal of the rewrite.]
- jcoffland 7y ago> Better architecture, but bug-for-bug compatible" probably isn't the goal of the rewrite. Why not? From there you proceed to fix the old bugs.
- scottrogowski 7y agoI've been a developer for almost 10 years and during that time, have been involved in small rewrites and major rewrites. None of them have gone well and there wasn't a single one that we didn't regret starting on. What you think is a mess of legacy code is often less messy than you think and a lot of the odd decisions were often made for very good reasons. When you rewrite a codebase, you often tend to over-engineer it for the vague goals of being "scalable" and "clean" but, while clean on the surface, it often comes out far more complicated and with the a similar number of bugs. Another way to look at it is you're taking what was previously an agile approach of iterative development and replacing it with a waterfall approach which sees a grand end goal (parity with legacy code). Software is complicated and there's no silver bullet. Adopting React over jQuery will not reduce complexity as much as you think. Adopting microservices over a monolith will not reduce complexity as much as you think. The only thing that will ever reduce complexity is careful iterative development of good abstractions which are inserted into the codebase one-at-a-time.
- jandrewrogers 7y agoRewrites, as opposed to refactors, are warranted when they address fundamental architectural limitations that hinder product success. There is a notable exception for cases where the software architecture effectively is the product and customers rely on observable behaviors good or bad e.g. RDBMS. Poor design is not the only source of architectural problems and it is not always avoidable; maintaining unused flexibility and optionality in architectures is often extremely expensive. In my experience, few companies rewrite systems with legitimate architecture issues. It is vastly cheaper in the short-term to redefine the business in terms of those limitations, which is what most actually do in these cases. The risk to this strategy, which I've seen manifest many times, is that a competitor without these limitations can change the expectations of the market and thereby render your product obsolete in surprisingly short order. Rewrites are always extremely expensive, along many dimensions, but they are also sometimes unavoidable and can lead to much greater product success than without. It is much more complex than the mere state of your code base. The calculus for whether or not it is worth it doesn't lend itself to simple analysis.
- bborud 7y ago1) Rewriting code is rarely, as the phrase suggests, about doing all the work over. Usually it is about restructuring the architecture and then adapting the old code gradually. 2) Throwing away code is not the same as throwing away knowledge. Provided that you didn’t lose the entire team, the knowledge is still there. The code is just the corpse of knowledge and it is worth just a fraction of what the team that wrote the code is. A rewrite is a good opportunity to revisit and review that knowledge. 3) With experience you learn to balance two forces: perfection and pragmatism. You should strive for perfection in everything you do, but you have to demand of yourself that you can release in a reasonable amount of time. 4) Every good software project I’ve been on has viewed the code as something temporary. This forces you to think about how you will replace every bit of the code as you write it. From this, workable architecture emerges unless you succumb to perfectionism. From your description your CTO’s instincts were right, but he might be missing the backbone to put you in your place. Your team needs a more experienced lead developer to help you guys mature as developers. And you need to pay less attention to articles and blogs, be less worried about what others think and ready to learn from someone more experienced. That’s going to be hard since we programmers tend to have huge egos. Even when we’re inexperienced.
- revskill 7y agoMy favorite strategy for code reuse is, make API configurable. That means instead of coding, you configure the system as much as possible. Only the core , which is the implementation details could be changed without affect the rest of system. One benefit/downside is that, whenever you change the core, or everything else works at the same time, or all thing breaks.
- anaphor 7y agoAm I the only one who will do an initial "proof of concept" implementation of something that I intend to rewrite in another language? E.g. I will write the "first draft" in Python or Racket because I know the language won't get in my way, and then if it needs to be redone in a lower level language or something that I'm less familiar with, then I have a reference implementation to compare it with. The same concept can even apply without switching languages if you just want a "naive" version that you know is correct.
- tolien 7y ago> Consider refactoring before taking a step to code rewriting There's got to be a Ship of Theseus argument here, where over the course of time you refactor the whole thing until none of the original code remains. I guess the difference between that and a rewrite is that after each refactor you should still be able to ship it.
- fatbird 7y agoI was the main programmer on a web application with 250k lines of code; it handled a single very complex product selection/configuration process. The time came to expand it to handle multiple products in the same family of products. We ended up, over six months, doing a complete rewrite that, from the outside, looked like refactoring (meaning, from the user's/product owner's perspective, the old stuff continued to work as we wrote the new stuff beside it, switching over as we went). We launched on time and on budget. My last week was a death march but that was it. There was now only 50k LOC. Where the previous version was tightly coupled to the product specification, we now had an engine that could perform the same selection/configuration process for any product that could be articulated in the new DSL we'd developed; as a bonus, the specifications were standalone modules so we could use them in completely separate web apps. The whole time we were aware that we were breaking the 'never rewrite' rules spelled out by Spolsky and others, but we were successful, the codebase took a huge leap forward both in speed of development and maintainability, and a subsequent project to add another set of products took half the time that was estimated. I don't know what we did right, exactly, except perhaps that our rewrite was never about burnout or annoyance at the existing codebase; our decisions along the way were always justified by what we had to deliver and a reasonable feeling of an experienced dev that refactoring the old code would take more effort than starting over. I'm not sure I buy the articles rejection of the rewrite they did, because he frames it entirely in terms of delay in updating and adding features, without seeming to consider that this was just legitimate technical debt that they had to pay off. The old code got them into a position where they could rewrite it. How many startups fail because their attempts to get it right the first time prevent them from ever launching and getting the customers that will pay to fix it later?
- beagle3 7y agoMost rewrites are unjustified, either because the old code is good enough, or because the guys in charge of the rewrite cannot deliver better (whether or not the old code is good enough). But some are. Arthur Whitney creates each version of the K language from scratch (he is now at version 7). They are not backward compatible, but every single one so far has been better on the metrics that he cares about (succinctness, speed, and revenue). Everyone says that the Mozilla->Firefox rewrite was unjustified, but ... the Mozilla codebase looked like a dead end to too many people working on it (from first and second hand accounts I heard), it's not clear that it was viable to continue with the old codebase -- and, even if it was, it is unclear that Mozilla would have been better off doing that: Microsoft was the unstoppable juggernaut playing dirty. IE won not based on technical merit (although it actually WAS the better one come version 4) - but arguably because it was integrated into the OS. In fact, it is quite likely that we would not have firefox today at all if they did not start the rewrite back then. C# is essentially a rewrite of Java; done for legal reasons rather than technical reasons, but IMO the result is a much better technical solution than trying to retrofit and comply with the (legal) legacy of Java. But ... yes, most rewrites are unjustified and often end in failure.
- kujaomega 7y agoI recommend reading about Domain Driven Design. For example reading the book: Implementing Domain-Driven Design by Vaughn Vernon