6 ms·
Sometimes it's best to be DRY right from the start. Several years ago, I did some contract work for a company that needed importers for airspace data and vario
by Stratoscope 2y ago
Sometimes it's best to be DRY right from the start.
Several years ago, I did some contract work for a company that needed importers for airspace data and various other kinds of data relevant to flying.
In the US, the Federal Aviation Administration (FAA) publishes datasets for several kinds of airspace data. Two of them are called "Class Airspace" and "Special Use Airspace".
The guy who wrote the original importers for these treated them as completely separate and unrelated data. He used an internal generic tool to convert the FAA data for each kind of airspace into a format used within the company, and then wrote separate C++ code, thousands of lines of code each.
Thing is, the data for these two kinds of airspace is mostly identical. You could process it all with one common codebase, with separate code for only the 10% of the data that is different between the two formats.
When I asked him about this, he said, "I have this philosophy that says if you only have two similar things, it's best to write separate code for each. Once you get to a third, then you can think about refactoring and making some common code."
That is a good philosophy! I have often followed it myself.
But in this case, it was obvious that the two data formats were mostly the same, and there was never going to be a third kind of almost-identical airspace, only the two. So we had twice the code we needed.
- eddd-ddde 2y agoThat sounds like it was as easy as copy paste. The day one of those formats changes slightly, it's really easy to update it. If you had a single code base, you now have to think how to refactor out the differences. In my experience when those differences appear is more likely to not be trivial.
- wnevets 2y ago> So we had twice the code we needed. Was that necessarily a bad thing and something that must be corrected for that code base? I usually follow the same rule of thumb until I find myself repeatedly updating both at the same time. If I can't update one without updating the other then they must be the same thing and its time to DRY. Don't Repeat Yourself when updating code.
- Stratoscope 2y agoGood points, thanks for bringing them up. Yes, there was some ongoing maintenance of this code where both versions had to be updated. The original author was not a pilot and was unfamiliar with some of the nuances of FAA airspace. One of the reasons they brought me in was that I am a pilot and knew how the FAA's data should be interpreted. In the end, not a huge deal, but it was annoying when I had to make the same changes in two places.
- thdc 2y agoKnowing to DRY there depended on business knowledge that the original author did not have. While they were wrong in this case, I would say it was a reasonable move to not DRY based on the code pattern itself at the time. And that's the big difference imo - DRYing based strictly on the structure of code vs business processes.
- chii 2y agoBut this implies that you have to guess when and where to DRY, which basically implies that there's no good way but via experience and domain knowledge! That's not what people want to hear - they want a silver bullet; a set of criteria for which DRY'ing could be determined from the onset!
- ebolyen 2y agoI don't know, that sounds like a complex kind of ingest which could be arbitrarily subtle and diverge over time for legal and bureaucratic reasons. I would kind of appreciate having two formats, since what are the odds they would change together? While there may never be a 3rd format, a DRY importer would imply that the source generating the data is also DRY.
- TeMPOraL 2y agoIn such case I think I'd go for an internal-DRYing + copy-on-write approach. That is, two identical classes or entry points, one for each format; internally, they'd share all the common code. Over time, if something changes in one format but not the other, that piece of code gets duplicated and then changed, so the other format retains the original code, which it now owns.
- ebolyen 2y agoI like that approach.
- Spivak 2y agoI've had the mantra "inheritance is only for code reuse" and it's never steered me wrong.
- jameshart 2y agoInheritance is only good for code reuse, and it’s a trick you only get to use once for each piece of code, so if you use it you need to be absolutely certain that the taxonomy you’re using it to leverage code across is the right one. All ‘is-a so it gets this code’ models can be trivially modeled as ‘has-a so it gets this code’ patterns, which don’t have that single-use constraint… so the corollary to this rule tends towards ‘never use inheritance’.
- Spivak 2y ago
- steve_adams_86 2y agoI find this is a case where different pipelines utilizing common functions in different compositions can be a great strategy. If something diverges and a function no longer makes sense in a pipeline, that’s not a big deal. Just pull it out and replace it with something bespoke that does the right thing. I’ve had a lot of success with this in embedded settings where data is piped into storage or OTA, and I want to format and pack it/send it up consistently but I might want to treat the data itself slightly differently.
- hansvm 2y agoA related concept that IMO still aligns with DRY is that you should only avoid seeming code duplication when things are _semantically_ the same. No matter the mechanism (codegen, generics, macros, inheritance, ...), if you can't give a concept a meaningful [0] name then you usually shouldn't DRY it up with any mechanism. Your example is a technique I also use a lot, but the critical point is that you're choosing to break out functionality which _is_ easy to name. [0a] More generally, I like a concept of "total" functions -- those which have sensible outputs for all their inputs. It's a bit of a tomayto/tomahto situation defining "all their inputs" (e.g., I'm personally okay using a function name like `unsafe_foo` and expecting a person to read the docs, and on the other extreme some people want sensible answers to anything the type system allows you to input), but the desired end-state is that when the project's requirements change you don't muck around with the ABI and implementation of `count_or_maybe_sort_for_these_three_special_customers_or_else_hit_the_db(...)`, or whatever much more generic and very wrong name the method actually has; the individual components are already correct, so you make the changes at the few methods which are actually wrong given the new requirements. [0b] Another way of thinking about it is whether the two things should always change in tandem. For two largely overlapping beaucratic data formats? Maybe; there's a comment somewhere in this chain suggesting that they'll never go out of sync, but I'm a bit paranoid of that sort of thing. For the particular data structures that are currently shared by those formats? Absolutely not; if one diverges then you can build the new structure and link it in. The old structure is still valid in its own right.
- magicalhippo 2y agoWe've got a large number of customer-specific file integrations, and a lot of them are indeed very similar as the customers have the same system on the other side. However almost all the time there's some tweaking needed. Customer A used field X for this but customer B used the field for that. So if a new customer comes and need an integration to a system we already support, even if we think they'll start out being identical, we just copy the code. Thing is, these things evolve. Suddenly we have to patch over some process-related issues in the other system for customer A, while customer B does not have that issue. Now we can fix A's integration without worrying at all about affecting B. Of course we write library and helper functions, and use those actively throughout, so we only repeat the "top level" stuff.
- klyrs 2y ago> Sometimes it's best to be DRY right from the start. Never zealously adopt a programming practice that can be summarized in a headline!
- OJFord 2y agoYeah, don't be DRY from the start, YAGNI.
- barfbagginus 2y agoI feel like the "sometimes" suggests that in most cases we should zealously not dry things, but in some cases we might want to. Doesn't that make you curious what cases those might be?
- klyrs 2y ago> Doesn't that make you curious what cases those might be? I've been programming for nigh on 40 years; I'm not terribly curious because I've seen successes and failures of pretty much every development paradigm firsthand. But since you ask, my comment above was a self-referential whimsicalism. I DRY my code in phases. A rule of 3 is decent (e.g., if you've copied a chunk of code three times, give it a think; if you've copied it 9 times, give it a good hard stare).
- PaulStatezny 2y ago> Sometimes it's best to be DRY right from the start. 3 things matter most in real estate: Location, location, location! 3 things matter most in programming: Context, context, context! DRY – like almost every other programming tool/paradigm/principle – are very often misused by a lack of the programmer's ability to discern correctly whether the tool/paradigm/principle fits in the specific context. It's not just a science. It's an art, too.
- klibertp 2y ago> It's not just a science. It's an art, too. It was... I will miss that until retirement, too, but the artisan part of this craft has been gradually dying for over a decade. I think this change started when "popular kids" started confidently saying they wanted to work with computers when they grew up. The effect is the proliferation of normies throughout the trade, now many of them with 10+ years of experience. The average developer's appreciation for the elegant and the inspiring grew weaker, and the idea of putting more work into a task than absolutely necessary (like, for example, stopping for a moment to consider the context before deciding on a tool or technique to apply...) lost all appeal. There's a thin line between aggressively pragmatic and ignorant, and the newer generations seem to treat crossing that line as a non-issue as long as the ticket can be presented as resolved. This mindset used to be confined to cubicles and neckties, but now it's seemingly everywhere... Don't mind me; I just feel unusually old and grumpy today...
- Stratoscope 2y agoOne time I worked with someone who said multiple times, "I closed more tickets in this sprint than anyone else on the team." What I learned from that is to create a ticket for every slightest little thing. Correct a typo in a comment? A ticket. Combine a tiny bit of duplicate code? A ticket. Add a little error check? A ticket. Forget working together to create a great and well-supported product. What counts is whether I closed more tickets than you!
- hoosieree 2y agoCome check out some of the array languages[1,2,3], or perhaps the retro-computing comfy vibes of a system like Decker[4]. Some of us still appreciate code as poetry. [1]: https://mlochbaum.github.io/BQN/ https://mlochbaum.github.io/BQN/ [2]: https://www.uiua.org/ https://www.uiua.org/ [3]: https://code.kx.com/q/learn/tour/ https://code.kx.com/q/learn/tour/ [4]: https://beyondloom.com/decker/tour.html https://beyondloom.com/decker/tour.html
- hakunin 2y agoDid the repetition become obvious before or after you saw both implementations? It’s possible that if you dried it up right away, you would’ve abstracted the wrong thing, and it’s way more obvious only in hindsight.
- Stratoscope 2y agoThis was some years ago, so my memory may be foggy - and I'm not instrument rated! As I recall, it was after I saw the two implementations. I got curious and looked at the original FAA data and specifications and saw how much the two kinds of airspace have in common.
- barfbagginus 2y agoHow was the dried code to own and maintain before and after? Was there pain before, and did it go away? Or did the operation prove to be more of a nitpick/bike shed? How long did you stick around with eyes on the code?
- Stratoscope 2y agoInteresting questions, thanks for asking! We never got that far. At that point, the code was what it was. Despite any possible design flaws, it worked. And dealing with FAA data that really never changes at all, once it worked, that was good enough. "If it ain't broke..." ;-) My job after that was to move on to other countries' airspace and obstacle data. So I tried to apply the appropriate amount of humidity on those.
- BurningFrog 2y agoI'm a fan of the "Rule of Three", but another great rule is to not mindlessly follow rules.
- heleninboodler 2y agoCall me crazy, but it's almost as if these should be guidelines considered by a thinking person, with experience to help inform them, rather than hard-and-fast rules that must be applied to everything.
- cjfd 2y agoThis happens when people follow rules mindlessly. 3 is an arbitrary number anyway. Even if 3 is the right number in most cases there will be cases where abstracting after 2 cases is best and others where abstracting after 4 cases is best, or any other number, really.
- jvans 2y agoThis is just being hindsight oriented. The way it worked out it was better to DRY from the start, but the person implementing it didn't know how the chips would fall. If you do what this guy did and are wrong, it looks like yes obviously it was the same thing and you should have DRY right from the start. If you DRY from the start and are wrong, someone writes a blog post about not DRYing from the start. It's a bet on which mistake is worse, because you will be wrong sometimes. DRYing from the start IMO is a worse mistake than duplication.
- Stratoscope 2y agoThat is a great insight. As always, "You don't know what you don't know." One time the guy who wrote the original code asked, "Why do they have towers at airports? Aren't the controllers down underground looking at radar screens?" It took two of us who were pilots to convince him that yes, the tower is up there so the controllers can see the airplanes with their own eyes. And yes, they are provided with binoculars. We all form mental models of the world around us. The best we can hope for is to update those models as we receive new information.
- tossandthrow 2y agoIt is easy to come in after the fact and say this. In reality: Had he DRYed it up from the beginning, you would probably have complained about a codebase that needs to corner cases into account deep in the code - the story had been turned around. Yes, it is annoying to get into an existing code base.
- baja_blast 2y agoBut you're refactoring an existing codebase so it's not prematurely dry. You know exactly how these classes diverge.
- gloryjulio 2y ago> Sometimes it's best to be DRY right from the start. In your case I don't see how it couldn't be refactored later on once you have the domain knowledge. In our company we always have 2 types of projects planned. We can't always ship new features under type 1. And refactoring is something we must do under the 2nd type to reduce the tech debt. With 2 type of projects in mind we always plan the capacities for both types of projects And that's how exactly how you should do dry. Once you see the repeated pattern, you should refactor it. I'm in the finance factor and we are seeing this kind of 'similar but not the same' problem a lot. We are constantly in the process onboarding new payment use cases while doing refactors to abstract away the common patterns.
- barfbagginus 2y agoIn some cases I would prefer to have two separate clear yet repetitive use cases, than to have, for example, a single abstract use case, that gets injected with two different factories at configuration time, depending on which sub case you want. In that case, reading and maintaining two simple use cases might be less work than reading an abstract use case, backtracking to the available factories, and then mentally interpreting the injection and factory behavior. Unless your abstractions really really make things just way simpler, being explicit could be better. Another place where this repetition tends to help readability more than it hurts maintainability is in test cases. Often abstracting things out with a little test fixture is helpful. But then being obsessive about this ends up making tests harder to maintain since there's all this long distance coupling that you constantly have to maintain. It seems that outside of test cases and use cases, we want to be much more diligent about DRY and picking the right abstractions - that makes logic in our use cases much simpler and more coherent. While inside the use case or test case, a little duplication of business logic is not so bad, and can actually improve the narrative of the code.