14 ms·

On DRY and the cost of wrongful abstractions

223 points by nkurz 10y ago

makmanalp 10y ago
More on the subject: http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstractio...
- mklim 10y ago
  > If you find yourself passing parameters and adding conditional paths through shared code, the abstraction is incorrect. It may have been right to begin with, but that day has passed. Once an abstraction is proved wrong the best strategy is to re-introduce duplication and let it show you what's right. I like this take on when go full DRY or not a lot. It's very easy to abstract out two things that look identical in their earliest implementation, but are actually different in intent and function--then when you get a new feature request on one of them they become obviously wildly different, and then to maintain the abstraction you have to throw in a bunch of conditional blocks into it and it turns into a nightmare to maintain. I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. It's just a rough rule of thumb to try and make sure you're not prematurely abstracting something and building the wrong thing, but anecdotally it's been useful for me.
  - stcredzero 10y ago
    > I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. I used to work in a shop with that rule, and I've mentioned it here on HN. It's still pretty easy to track down and rewrite 3 occurrences. Heck, it's still pretty easy to track down and rewrite 7, though as you let the numbers increase, you run the risk of missing an occurrence, making a mistake when you do the refactor, or the introduction of a confounding "idiosyncracy" in one of those occurrences. In my experience, 2 occurrences is too little data to justify refactoring around, unless those are pretty hefty chunks of code. 2 occurrences of 2 consecutive lines is definitely too little.
    - rdiddly 10y ago
      Exactly - 3 occurrences is a pattern - 2 is bordering on zealotry.
    - dllthomas 10y ago
      I think the focus on counts is wrong. The first question is "are these the same for a reason." If there are two places in the code that need to behave the same or the code is broken, then you should probably pull it out. If there are 100 places in the code that happen to behave the same, but any subset of them could change independently tomorrow, then you might not want to pull it out.
  - gregmac 10y ago
    > I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. It's just a rough rule of thumb to try and make sure you're not prematurely abstracting something and building the wrong thing, but anecdotally it's been useful for me. I've not heard that before, but I like it. I have always advocated for waiting until there's at least two instances, with a similar reasoning: don't abstract until there's a 100% proven need to do so. > It's very easy to abstract out two things that look identical in their earliest implementation, but are actually different in intent and function--then when you get a new feature request on one of them they become obviously wildly different On top of waiting for the proven need, I think the other big thing you should strongly consider when building an abstraction is the future of that code. If you need to make a change (regardless of what that change is), are you fairly certain every instance of it going to change in the same way? If the answer is not an immediate and resounding 'yes', then you should really stop and consider if your abstraction is actually a helpful step towards DRY, or merely clever "architectural astronautism" that is only going to be a pain to maintain later.
- twic 10y ago
  Does this article add anything to Sandi Metz's original?
raymondh 10y ago
I agree with this important article. In the large open source project I work on, we have a prolific contributor who is systematically going through the entire code base, "refactoring" some of it in way the impairs my ability to read, understand, and maintain the code (even code that I originally authored). Over time, we've added scores of new macros -- as result ordinary C knowledge is now insufficient to read and understand what the code is doing. As the author suggests, the guiding principle should be factor only when it results in a net reduction of complexity. In some cases, the cost of adding new abstractions outweighs the gains from Don't Repeat Yourself. There is some value in having some blocks of code that can be read in isolation, even if there some bit of repetition between blocks.
- blazespin 10y ago
  The problem likely isn't DRY, the problem here is poor abstractions.
  - wruza 10y ago
    The problem here is that our so-called editors cannot inline calls for us.
    - smt88 10y ago
      Many editors let you peek at the source of a call in a pop-up window.
      - wvenable 10y ago
        Even so, it sort of breaks the flow. The difference between reading a novel from beginning to end or a choose-your-own-adventure book.
        smt88 10y ago
        In a large app, inlining all calls would be unreadable. Having sane, descriptive names for functions (and using functional programming, for that matter) solve the problem better than inlining.
svantana 10y ago
I am firmly in the DRY camp, although obviously there are limits to how far you should go to maintain the dry-purity. In my experience, the biggest issue with copy-pasting code is that someone else (or your future self) will edit the code in only one of the places, making the codebase inconsistent. If abstraction gets too costly, I have found it useful to simply add a comment in both places, "copied from x" and "pasted from y" respectively. Or, if there are constants involved, break those out as global constants.
blazespin 10y ago
I would have taken it more seriously if he addressed security and unit tests. His perspective here is sophomore. He is right, but not because of what he wrote. DRY isn't just about abstractions, it's about the cost of writing new code. And testing and security verification are hidden costs to duplicated code.
- dang 10y ago
  > His perspective here is sophomore. An HN comment usually gets surprisingly better if you take the name-calling bits out of it. I'd say that's the case here.
  - hueving 10y ago
    I understand that it was a bit blunt, but isn't referring to a perspective as 'sophomore' just another way of saying it was short sighted or that it doesn't take the broader ramifications into account?
    - dang 10y ago
      Yes, but to my ear it also includes a putdown, along the lines of 'juvenile' or 'childish'. That's why I referred to it as name-calling.
wruza 10y ago
If we could track copy-pasted chunks, then manual abstraction step would be needless, as it is already clear from editing history. We could then transform between abstract-nonabstract source or completely remove that distinction by making it view property. And modify one place and quickly update (or not) all others. We'll also have to deprecate 'modern' text sources and store code in structured way. That should be all obvious to anyone with 10+ programming experience, imo, but I always see such "oh, dry or not, kiss or not" jitter. We now have great tools to do everything, except programming. Someone must make programming app better than just notepad-with-word-completion junk.
- woah 10y ago
  Ever heard of a function? If it's as simple as copy paste, that's all you need. The trickier abstractions come when you need something to be almost the same in two places. Copy and paste tracking wouldn't work there. Replacing functions with some kind of copy paste tracking editor does not sound like an improvement.
  - taeric 10y ago
    Coverity actually has a really good tracking ability where it will find code that was copy/pasted/edited but the edit looks wrong. I can't remember the name, but it will identify what it thinks is the original, and show where the copy/paste is, along with the edit it thinks was missing. This was usually around some error handling logic in our code base. Think: if (checkIsValid(foo)) { printf("Looks like foo is not valid. foo == %s\n", foo); } Wasn't uncommon for someone to copy that and forgot to change all places "foo" appeared. Long story short, advanced static analysis has come a long way.
  - wruza 10y ago
    This is already solved on bigger scale -- RCS. You can develop, merge, blame, diff slightly different filesets, but not lineranges. Look, we already have both 'just functions' and dry-vs-complexity problem unsolved.
- jonaf 10y ago
  Static code analysis and a good IDE make this point considerably less important. For example, duplicating code is not nearly as expensive in my Spring (Java) application as it is in that shiny Nodejs webapp. It's still important, but solid tooling combats it somewhat.
seanhandley 10y ago
TL;DR Think before you type.
elcct 10y ago
High level of abstractions accumulate especially in big monolithic applications and I agree it takes time to get the feature in. Especially, you have to get an idea of what code has already been written and how it plays with the components before you start doing anything. First, to not repeat yourself and also to lessen the chances of doing something wrong. I think everything has its trade-offs and use cases. This approach in large applications could make you write less tests, as there is less amount of repeated code. On the other hand you can split your project into independent smaller parts and at the cost of sacrificing DRY you can get something more easy to understand, but then you have to put more effort on testing and integration testing to see if all bits play together correctly and as expected. I agree that the sense of what should be abstracted and what shouldn't comes with experience. It is also very important to get as much information about the problem - this helps choosing the right approach in the context of above.
- brightball 10y ago
  I tend to take DRY more from a standpoint of data and code to transform that data into other data. I'm less concerned about duplication in code that uses data than I am the code that determines whether the data itself is correct.
bpchaps 10y ago
My complaint with DRY - sometimes it's just significantly easier to learn a codebase or learn the problems that led to its structure/abstraction by just writing many of the underlying components yourself. Whether that code goes into the codebase is somewhat irrelevant from a learning perspective. On top of that, when 95% of the code you work with is based on a "Minimal Viable Product" (the bane of my existence), assumptions will probably be made incorrectly, or in a way that adds significant technical/organizational debt. By doing it yourself, the original assumptions can be put into a different light and can help reduce all sorts of debt in the future.
catnaroek 10y ago
The core of the problem is that people often confuse “parameterization” with “abstraction”: Parameterization is simply adding adjustable knobs. For example, a function can take as argument a gigantic list of flags, where each particular combination of flags makes the function do something slightly different. Abstraction is actually separating concerns. It's dividing a program in logically independent parts, whose correctness can be established individually, without worrying about the others. Abstraction means that software components only interact with each other through explicitly designated interfaces, often enforced by the programming language itself. Parameterizing is easy. Abstracting is difficult. Unfortunately, parameterization without abstraction is a source of headaches in the long run.
- taeric 10y ago
  There is a quote from Pólya that I love in margin notes of Concrete Mathematics. There are two kinds of generalizations. One is cheap and the other is valuable. It is easy to generalize by diluting a little idea with a big terminology. It is much more difficult to prepare a refined and condensed extract from several good ingredients. I think it applies here, fairly well. Just finding a way to not repeat yourself is relatively easy to do. Finding a way to not repeat yourself, but keep the same level of information conveyance is actually quite difficult.
sghiassy 10y ago
Thanks for the article. I agree with its main points
jstimpfle 10y ago
Another thing that is difficult to get right is choosing a suitable font...
jnordwick 10y ago
I am firmly in the DRY camp with a very important caveats: 1- Do no generalize so you can factor out like code too early. The first time just write it. The second time write it again even if only minimally different. The third time is a judgement call. By the fourth time you see the proper abstraction, something that might have been much different if you were to have refactored early on. Corollary to #1- Don't run ahead of yourself making all these placeholder functions, interfaces, and abstract classes with the expectation that you would have had to refactor this way later. Unless you done versions of your project a dozen times (even then, sometimes I don't even do it), you are most likely going to be wrong. 2- Don't refactor out single lines unless there a lot going on on that line or you have the experience to know (don't lie to yourself either) that this will be an important and growing line. 3- Sometimes performance can trump. And sometimes you do know this ahead of time, especially if performance is correctness as it is on some projects with very tight constraints. Besides that. I freaking hate seeing similar code everywhere, blowing up my instruction cache, making changes difficult. Please stop it.
- InclinedPlane 10y ago
  I disagree with your point 2 very strongly. A lot of people have a mental block when it comes to 1-line functions, and they really shouldn't, they can often be very helpful for a lot of reasons. Generally, the most useful tool in refactoring is to use extract method relentlessly, even into 1-line functions. More often than not, a well named function with well named parameters is one of the absolute best and most reliable ways to make your code as close to "self-documenting" as possible. But there are additional advantages as well. Spinning things off into function calls often makes refactoring easier. Because it discourages bad habits like local variable reuse (using variables as scratch pads) and excess mutable state. It also encourages a more functional style, which has a lot of benefits especially in multi-threading but also in the ability to reason about and determine the correctness of code. It can also make it a lot easier to see when you need to spin off functionality into other classes/modules. For example, if you have a member field and you have a ton of little one line snippets everywhere that anything happens with that field, that usually means you need to extract it out into a class and have those one-line snippets be various methods on that class. When all that code is inline, this can be harder to see. Additionally, there are lots of mental blocks that happen when you have inline "one liners". There's a bias to avoid "cluttering them up", even when it's necessary. For example, when you need to add error checking and error handling. When you've spun out the code into a method call, you tend not to have that block, you just add the error handling code because it's trivial, and going from a 1-line method to a 5-line method isn't a big deal. But bloating up another function by taking a sweet one-liner and replacing it with 5-lines of code can seem like clutter, so often times you just have an aversion to doing so. Additionally, sometimes those "sweet one-liners" can be more readable when they're broken up into multiple lines, and this is again something that is trivial and straightforward when it's spun out into its own method but will see resistance when the code is inline. This is true even if the code remains a single statement broken up into multiple lines to enhance readability. A lot of the time these patterns aren't obvious. Because the end result of spinning off one-liners is often not one-line method definitions, it's short methods that are a couple lines with several lines of extra error detection/handling. Or it's separate utility classes to encapsulate handling of special member fields. The idea of ensuring that you always have "meaty" functions is unhelpful, what's important is thinking about things from a perspective of abstraction, modularity, readability, and correctness.
leovonl 10y ago
The DRY principle issues usually apply very well to OO programming and the whole "framework" concepts, where individual libraries or pieces of the code cannot easily be reused for different projects. Another big issue in the framework case is the inversion of control, which further restricts how the reuse can happen and makes it very hard to reuse the code differently in an efficient way.
jimjimjim 10y ago
The idea of 'don't repeat yourself' is an ok guideline unless, like most proverbs/sayings/mottos/slogans it gets used as a absolute rule. feel free to ignore it, mock it and toss it aside if it leads to bad abstractions, highly convoluted structures or write-only code. anyone that has had to do maintenance or adding features to a large OO codebase from the early 2000s will have seen vastly massive class hierarchies where following the path of execution is a roller coaster ride through 10 files for what could have been a 30 line function. these days grep/vim/sublime/emacs/and the rest of the gang all have power search and replace editing functions. sometimes it's better for the whole world to just copy-paste-alter your code.
- stcredzero 10y ago
  an ok guideline unless, like most proverbs/sayings/mottos/slogans it gets used as a absolute rule. Amen! a roller coaster ride through 10 files for what could have been a 30 line function. sometimes it's better for the whole world to just copy-paste-alter your code. Then what happens when you have almost a hundred copy/pasted slightly rewritten 15-30 line variations on the same theme? How do you refactor then? (Yes, I have seen this is production systems, and yes, it was very critical code!) As you say, it comes down to cost/benefit. Basically, you just have to keep potential refactoring/rewrite costs down so that you are never trapped. Caveat: You can seldom predict the risks as well as you think you can. What you can depend on, is being observant to historical patterns in your codebase. It's hard to predict the business needs and the architecture in the future. On the other hand, it's often quite easy to see the historical trends in the codebase. Really, the analogy of the physical file room is a great one. (Or if you're not familiar, a library, a tool chest, or any kind of physical inventory system works too.) You can have a file clerk that seems like "super-filer" because he never "wastes" time by putting files back, but this never works out in the long term. The same goes for a filing staff that never reorganizes the file room. Also, one can generally see the long term disaster developing long before it results in the dramatic disaster. You can see which shelves are getting filled up, and which drawers are getting overstuffed, usually weeks or months ahead of time.
  - Asooka 10y ago
sysk 10y ago
Am I the only one who thinks this sort of advice would be immensely more useful with some concrete examples?
- Cyclone_ 10y ago
  I was thinking the same thing, it's hard to judge this article without knowing exactly how much code duplication the author thinks is appropriate and in what circumstances.
stcredzero 10y ago
Yes, you read that right. Duplicating code (a.k.a copy + paste) can be a good thing. Namely when the abstraction that would replace repetitive portions of your codebase is a pain to understand. Yes, this 1000 times! DRY or any pattern/principle is not some sort of universal truth/declaration of ultimate goodness from your deity or some deep law like the Heisenburg Uncertainty Principle! It's just a rule of thumb. As a practical technologist, you must always implement with respect to the contextual cost/benefit. Taking any principle as an automatic ultimate good is simply intellectual laziness! I was in a company with a Java product and a Smalltalk product. The Smalltalk product was rather well factored. We had a policy of rewrites to keep things architectually clean. We had the fantastic Smalltalk Refactoring Browser as our IDE, and we weren't afraid to use it. (To this day, you can refactor an order of magnitude or more faster with it than you can in other languages.) The Java product consisted of many independent parts with tons of duplicated code between them. Definitely not DRY. (There were also 20X more Java programmers than Smalltalk, for a comparable product.) One thing I noticed, is that bugs on the Java side were more prevalent, but never affected the entire application. Whereas, the far better factored Smalltalk product could occasionally have the rug pulled out from underneath the entire application by the introduction of a library bug. (Though it would be very rapidly fixed.) Another thing at that shop: We had a policy of only empirically verified DRY. We never wrote code to DRY in anticipation: only in response to actual code duplication. Generally, actual practice and experience is better at producing exactly the needed architecture than the prognosticating cleverness of programmers. EDIT: Shower thought! (Yes, I literally got out of the shower just now.) There is a lot of emphatic but somewhat thoughtless application of principles and rules of thumb because it's actually motivated by signalling. The priority is actually on the opportunity to signal and not on the underlying cost/benefit calculation. There is a nice analogy for this in music. Once a musical scene gets a certain degree of prominence and/or commercial success, people start prioritizing sending the signal that they are part of that musical scene. So you will find bands and musicians using those techniques and stylistic flourishes, not because of the underlying aesthetic principles in support of the particular piece of music, but as an opportunity to signal. This happens in OO and also in the startup scene. (And, OMG, but is signalling given a lot of energy in the Bay Area!) So the insider move is this: Look for the underlying motivation behind the signal. Did this person jump on the opportunity to signal, or did they first seek out the information to make the determination of cost/benefit? How carefully did they do that? (Hopefully, we can get some of the hipsters/pretenders to at least go through the motion of doing the above. Even just going through those motions turns out to be beneficial.)
caseymarquis 10y ago
Article in a nutshell: Goodness = DRY×KISS
SideburnsOfDoom 10y ago
Yes, this is known as "Beware the Share" http://programmer.97things.oreilly.com/wiki/index.php/Beware_the_Share http://programmer.97things.oreilly.com/wiki/index.php/Beware...
RandomInteger4 10y ago
New Motto: Don't Repeat Yourself, Unless You Need To Repeat Yourself, In Which Case Repeat Yourself, But Be Careful How You Repeat Yourself ... DRYUYNTRYIWCRYBBCHYRY
- zachrose 10y ago
  Dry, un-try, wicker baby cherry.
gsmethells 10y ago
Figure out what varies => encapsulate that. Could be behavior, structure, or creation. This idea that you can avoid DRY in a positive way simply means one has yet to recognize that first statement.
sbov 10y ago
I've come to the conclusion that there are at least 3 kinds of DRY, and application varies: 1) There's a higher level abstraction in your code. Abstract at your own risk, unless you have at least 3-4 instances, or your architecture absolutely requires it. 2) There's wrappers around libraries and you only use it a certain way. I'm OK with this here. Sure, I could copy-paste the same exact parameters each time I use it in my code, but I will just write my own fiddleFoo() and use that every time. 3) There's when things MUST BE THE SAME. You should abide by DRY even if you only have 2 instances. E.g., we have an SOA and we DRY our routing. Routing MUST match or else things break.
fpoling 10y ago
DRY is rather language-dependent. In Java, where only single result can be returned from a function, it is often not worth the efforts to factor out common functionality if that requires adding an extra class just to pas few in/out parameters. In C++, on the other hand, the barrier to follow DRY is rather low as a I can literally move code to a helper function and easily parametrize it for reuse in another place.
toomim 10y ago
Here's a research article I wrote on this topic; there's a section on the "costs of abstraction": http://harmonia.cs.berkeley.edu/papers/toomim-linked-editing.pdf http://harmonia.cs.berkeley.edu/papers/toomim-linked-editing...
- eternalban 10y ago
  Interesting. Does it scale? (Skimmed the paper and believe have understood the process but examples you have are limited to n=2 clones. What if n>>2?)
wpietri 10y ago
I think this article could benefit from a few good examples. Sure, one should never apply principles in a rote fashion; they're guides to judgment. But the way we develop good judgment is through practice. Without seeing how he'd apply this in specific situations, I don't feel like I've really learned anything.
bogomipz 10y ago
I think the title should be "wrong abstractions." Wrongful means unfair/unlawful/unjust but it doesn't ever mean incorrect.
z3t4 10y ago
Something that I see more and more of lately is module exaggeration, like projects that have tens of thousands of small files that all depend on each other and are complexed together using modules. There are 5-10 imports then a few lines of code that binds them together and finally an export ... The worst part is that this is done by respectfully software companies and even taught in guides and tutorials.
breatheoften 10y ago
"It can be expensive to use the wrong abstractions, therefore you should copy and paste" is not sound reasoning. The _right_ abstractions can be incredibly (insanely, ridiculously) beneficial to a code base across all dimensions -- correctness, maintainability, performance. Searching for the right abstractions is definitely worthwhile - even if it sometimes incurs extra rewrite/refactor churn. The better the abstractions used, the easier the refactoring, the faster you find the good abstractions, the easier it is to change the code as your understanding of the problem grows ...
- rdiddly 10y ago
  He's saying duplication can sometimes beat inappropriate abstraction. You're saying appropriate abstraction beats both duplication and inappropriate abstraction. (Which I thought was obvious.)
- dllthomas 10y ago
  Sometimes what you have is a coincidence, not an abstraction. In that case, you should duplicate the coincidentally-identical implementation.
dllthomas 10y ago
DRY is a problem because it's so catchy and seems so self-explanatory. The original formulation of DRY spoke in terms of information. Any piece of information should exist in exactly one place in your code. As I've said before, modulo some caveats related to double-entry bookkeeping, this seems exactly right. An inveighing against anything that looks repetitive is not. The goal is not compression, it's ease of understanding and ease of making correct modifications. I've started using "Huffman coding" to refer to the practice of misapplying DRY by deduplicating superficial similarity rather than information content.
- zachrose 10y ago
  This is a cogent way to frame the acronym. There are times when repeating yourself is just accidental. I might have the same middle name as a coworker. Do we actually "share" that middle name? That depends on who's asking, or what system you're trying to build. But probably not.
Rygu 10y ago
For a more visual explanation of the "cost of abstractions" watch this presentation by Cheng Lou (Facebook). https://www.youtube.com/watch?v=mVVNJKv9esE https://www.youtube.com/watch?v=mVVNJKv9esE
- InclinedPlane 10y ago
  This is an excellent talk which introduces some very solid concepts and breaks down the problem in a very sophisticated way. I hope more people watch this. In fact, I think this talk is superior to the linked article. You should consider submitting this talk directly to HN.
cle 10y ago
My own experience with DRY is that it has diminishing value in a distributed systems ecosystem. There is definitely still value, but code coupling between components of a distributed system is very costly. I'm pretty fed up with coordinated deployments, big bang upgrades, etc., so I'm very conservative about applying DRY. Often I'd rather copy a bit of code to avoid the dependencies, which usually pays off better in the long run.
smnplk 10y ago
I think we need to address this giant elephant in the room called object oriented programming. https://www.youtube.com/watch?v=IRTfhkiAqPw https://www.youtube.com/watch?v=IRTfhkiAqPw
z3t4 10y ago
DRY etc should just be guidelines, not rules. I hate when something is made overly complex just to avoid repeating, global variables, long functions or large files. For example complicated one liners in a imperative language.
deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
pm24601 10y ago
Easily solved: When writing an abstraction, document with a date the reason for the abstraction. Seriously, a comment that goes like this: "2009 sept 20 : Peter Pan: this abstraction is to hid the ugliness of using SQL Server 2003 which is still in use by Slow Corp." ... now in 2016 a maintainer knows the assumptions and the reasons for the abstractions and can make a much clearer decision about how the abstraction should be treated.
coldtea 10y ago
Off topic, but since somebody will always complaint about the bad layout or the difficulty of reading of a blog post submitted to HN, I'd like to point out the excellent readability of this one.
- firloop 10y ago
  The stark blacks kind of burned my eyes, but I definitely agree it was readable.
  - coldtea 10y ago
    One can always lower the brightness of their screen, but they can't easily change the contrast in the font/bg. Besides, research has shown that plain ole black on white is the best combo for readability.
jayvanguard 10y ago
This article needs to be shown to all upcoming architects, senior developers, and design minded engineers. Too often the balance between abstraction and specialization isn't thought about explicitly but rather dogmatic rules kick in.
agentultra 10y ago
All too often the kind of "abstractions" we talk about in programming are the leaky, code-indirection kind. These abstractions tend to proliferate in a single code-base over generations of programmers. It leaves one wondering if these thousands of lines of code actually compute anything. In my experience the software at an organization is often a reflection of its culture. Thick with leaky abstractions, interfaces, facades, and indirection? Years of frustrated mediocre programmers, each slightly confused by the previous generations' shenanigans, led by a clueless management team that care more about deadlines than deliverables. There are signs that people are trying to do better but the code takes years to show improvement as it resists change at every corner. A crystalline entity designed and driven by data, never doing more work than is required? A team that has had little churn and is led by passionate engineers often more concerned with deliverables -- at least at first, because it's often fairly easy to make changes when your code-base adheres to a well-defined algebra, is thoroughly tested, and even has a well-formed specification of its critical components. DRY is as good an idiom as "measure twice, cut once." The difficulty is that software is often more complex to understand than a cabinet. However I can't slight it for being a bad idiom because of the wide variability in skill for such a difficult task as programming. We need something to teach new programmers that at least get them thinking about the abstractions they wreak havoc with.
- thr0waway1239 10y ago
  Your comment reminded me of Edmond Lau's book - The Effective Engineer [1] - where he talks about putting in a good amount of effort into the onboarding process for new engineers. His premise - having a senior engineer spend an hour a day for the first month helping the new employee with explaining the existing abstractions being used, the underlying design of various systems, etc. - would still be only about 20 hours, which is still only 1% of the number of hours that employee will spend in their first year - about 2000 hours. As a result, I believe that armed with that knowledge, the new employee is likely to be much more productive, failing which, at least cause less damage to the code base. I would say that the first example you mention - leaky abstractions et. al. - are just as much (or maybe more) due to poor onboarding as they are due to the frustration of mediocre programmers. There is a lot to be said for good process, which software engineering as a discipline falls short of quite consistently. [1] https://www.amazon.com/Effective-Engineer-Engineering-Disproportionate-Meaningful/dp/0996128107/ https://www.amazon.com/Effective-Engineer-Engineering-Dispro...
kazinator 10y ago
Any time you repeat some block of code twice, you should religiously turn it into a function which is called twice with the same arguments. Any time you call a function twice with the same arguments, you should religiously write a two-iteration loop around just one call. Any time you have more than one two-iteration loop in the same program, you should write a 'dotwice' macro and use that instead. Any time you have a function call sitting in a loop just for the sake of being repeated, you should avoid the argument expressions being evaluated multiple times; you must religiously evaluate the arguments to temporary variables outside of the loop, and in the looped function call, refer only to the temporaries. In order not to repeat this pattern itself you need a proper "calltwice" macro, and to be working in Lisp, ideally: (calltwice yourfunc expr ...). yourfunc and each expr are evaluated once, then the resulting values are applied to the function twice. This is a DRY town, damn it!
lambdabit 10y ago
Can we just ban coding posts that have no code? They usually give you an illusion you learned something but are a total waste of time. It's like reading a self-help post on following your biggest dreams or something. The hard part isn't these big beautiful ideas, it's how to actually put it in practice. How to actually take something away and measure whether abstractions or duplications are more costly in real code??
- threepipeproblm 10y ago
  This seems like an anti-intellectual argument. For example would "ban" many of Dijkstra's famous numbered papers. I'm guessing you haven't read them. While we're at it, why don't "we" just "ban" - mathematical proofs without numbers - all criticism, as it's about art but contains no art - all papers in the field of music theory, since they aren't music.
  - lambdabit 10y ago
    It's definitely anti-intellectual. But people who run this site themselves say they're interested in high signal to noise ratio. I don't think those are good comparisons. Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example.
    - threepipeproblm 10y ago
      >> Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example. Oh, you mean like some of Dijkstra's numbered papers? https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1012.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E... I think "examples might help" is valid feedback, but the author has sort of provided a reason why examples aren't included: in small projects the abstractions are usually worth it, was my read. So while the author could talk about examples, he probably couldn't include then in a piece of this size. "Can we ban" is what has rubbed me wrong way. Who is "we". How are "we" "banning"? Does this just mean you don't agree with this post being highly upvoted? Or that you want to circumvent the opinions of those who upvoted with a "ban"? It is interesting also to think about the irony of an argument that one should work in terms of concrete examples of the subject matter, in objection to a piece that argues one should not always abstract things.
pyronik 10y ago
Sometimes I think programmers and theologians have a lot in common. Both have dogmas and articles of faith which are backed by internally consistent logic. Now to the comments to hear the Catholics and Protestants debate!
kefka_p 10y ago
Frankly without encapsulations inside of encapsulations, the author would have never been able to make this blog post. Reference TCP packets wrapped inside of Ethernet 2 frames as the basis of the entirety of the internet. I don't think this is as "extreme" as some perceive it.
buzzybee 10y ago
I was younger and I thought that the right abstraction - and more specifically, the right language feature - would surely lead me to higher productivity. By invoking a certain keyword my code would magically be better in some way. However, language features are not the only abstractions you have available. In fact, they're typically the least interesting ones you could apply to a problem - any yokel can lift code out and slap parameters, annotations, reflection, or generic type signatures on it. But to reach interesting abstractions, you have to be patient enough to do discover the shape and flow of the code over time and subsequently apply exactly the right data structures and algorithms to either reproduce the same shapes in an abridged form, or different shapes that achieve the goal more efficiently. So when I position myself as anti-DRY, which I often do now, I'm saying, don't succumb to the thinking that just shovelling things around and slightly repackaging them will add up to what you wanted. An actually useful abstraction cuts so deep that it may amount to a different thing altogether. The language features are there to solve very straightforward situations that are known to reoccur time and again. Do not use them in clever ways.

223 points by nkurz 10y ago

More on the subject: http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstractio...

mklim 10y ago
> If you find yourself passing parameters and adding conditional paths through shared code, the abstraction is incorrect. It may have been right to begin with, but that day has passed. Once an abstraction is proved wrong the best strategy is to re-introduce duplication and let it show you what's right. I like this take on when go full DRY or not a lot. It's very easy to abstract out two things that look identical in their earliest implementation, but are actually different in intent and function--then when you get a new feature request on one of them they become obviously wildly different, and then to maintain the abstraction you have to throw in a bunch of conditional blocks into it and it turns into a nightmare to maintain. I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. It's just a rough rule of thumb to try and make sure you're not prematurely abstracting something and building the wrong thing, but anecdotally it's been useful for me.
- stcredzero 10y ago
  > I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. I used to work in a shop with that rule, and I've mentioned it here on HN. It's still pretty easy to track down and rewrite 3 occurrences. Heck, it's still pretty easy to track down and rewrite 7, though as you let the numbers increase, you run the risk of missing an occurrence, making a mistake when you do the refactor, or the introduction of a confounding "idiosyncracy" in one of those occurrences. In my experience, 2 occurrences is too little data to justify refactoring around, unless those are pretty hefty chunks of code. 2 occurrences of 2 consecutive lines is definitely too little.
  - rdiddly 10y ago
    Exactly - 3 occurrences is a pattern - 2 is bordering on zealotry.
  - dllthomas 10y ago
    I think the focus on counts is wrong. The first question is "are these the same for a reason." If there are two places in the code that need to behave the same or the code is broken, then you should probably pull it out. If there are 100 places in the code that happen to behave the same, but any subset of them could change independently tomorrow, then you might not want to pull it out.
- gregmac 10y ago
  > I heard a rough rule of thumb to stay away from abstracting anything until you see it repeated identically at least three times, not two. It's just a rough rule of thumb to try and make sure you're not prematurely abstracting something and building the wrong thing, but anecdotally it's been useful for me. I've not heard that before, but I like it. I have always advocated for waiting until there's at least two instances, with a similar reasoning: don't abstract until there's a 100% proven need to do so. > It's very easy to abstract out two things that look identical in their earliest implementation, but are actually different in intent and function--then when you get a new feature request on one of them they become obviously wildly different On top of waiting for the proven need, I think the other big thing you should strongly consider when building an abstraction is the future of that code. If you need to make a change (regardless of what that change is), are you fairly certain every instance of it going to change in the same way? If the answer is not an immediate and resounding 'yes', then you should really stop and consider if your abstraction is actually a helpful step towards DRY, or merely clever "architectural astronautism" that is only going to be a pain to maintain later.
twic 10y ago
Does this article add anything to Sandi Metz's original?

raymondh 10y ago

I agree with this important article. In the large open source project I work on, we have a prolific contributor who is systematically going through the entire code base, "refactoring" some of it in way the impairs my ability to read, understand, and maintain the code (even code that I originally authored). Over time, we've added scores of new macros -- as result ordinary C knowledge is now insufficient to read and understand what the code is doing. As the author suggests, the guiding principle should be factor only when it results in a net reduction of complexity. In some cases, the cost of adding new abstractions outweighs the gains from Don't Repeat Yourself. There is some value in having some blocks of code that can be read in isolation, even if there some bit of repetition between blocks.

blazespin 10y ago
The problem likely isn't DRY, the problem here is poor abstractions.
- wruza 10y ago
  The problem here is that our so-called editors cannot inline calls for us.
  - smt88 10y ago
    Many editors let you peek at the source of a call in a pop-up window.
    - wvenable 10y ago
      Even so, it sort of breaks the flow. The difference between reading a novel from beginning to end or a choose-your-own-adventure book.
      - smt88 10y ago
        In a large app, inlining all calls would be unreadable. Having sane, descriptive names for functions (and using functional programming, for that matter) solve the problem better than inlining.

svantana 10y ago

I am firmly in the DRY camp, although obviously there are limits to how far you should go to maintain the dry-purity. In my experience, the biggest issue with copy-pasting code is that someone else (or your future self) will edit the code in only one of the places, making the codebase inconsistent. If abstraction gets too costly, I have found it useful to simply add a comment in both places, "copied from x" and "pasted from y" respectively. Or, if there are constants involved, break those out as global constants.

blazespin 10y ago

I would have taken it more seriously if he addressed security and unit tests. His perspective here is sophomore. He is right, but not because of what he wrote. DRY isn't just about abstractions, it's about the cost of writing new code. And testing and security verification are hidden costs to duplicated code.

dang 10y ago
> His perspective here is sophomore. An HN comment usually gets surprisingly better if you take the name-calling bits out of it. I'd say that's the case here.
- hueving 10y ago
  I understand that it was a bit blunt, but isn't referring to a perspective as 'sophomore' just another way of saying it was short sighted or that it doesn't take the broader ramifications into account?
  - dang 10y ago
    Yes, but to my ear it also includes a putdown, along the lines of 'juvenile' or 'childish'. That's why I referred to it as name-calling.

wruza 10y ago

If we could track copy-pasted chunks, then manual abstraction step would be needless, as it is already clear from editing history. We could then transform between abstract-nonabstract source or completely remove that distinction by making it view property. And modify one place and quickly update (or not) all others. We'll also have to deprecate 'modern' text sources and store code in structured way. That should be all obvious to anyone with 10+ programming experience, imo, but I always see such "oh, dry or not, kiss or not" jitter. We now have great tools to do everything, except programming. Someone must make programming app better than just notepad-with-word-completion junk.

woah 10y ago
Ever heard of a function? If it's as simple as copy paste, that's all you need. The trickier abstractions come when you need something to be almost the same in two places. Copy and paste tracking wouldn't work there. Replacing functions with some kind of copy paste tracking editor does not sound like an improvement.
- taeric 10y ago
  Coverity actually has a really good tracking ability where it will find code that was copy/pasted/edited but the edit looks wrong. I can't remember the name, but it will identify what it thinks is the original, and show where the copy/paste is, along with the edit it thinks was missing. This was usually around some error handling logic in our code base. Think: if (checkIsValid(foo)) { printf("Looks like foo is not valid. foo == %s\n", foo); } Wasn't uncommon for someone to copy that and forgot to change all places "foo" appeared. Long story short, advanced static analysis has come a long way.
- wruza 10y ago
  This is already solved on bigger scale -- RCS. You can develop, merge, blame, diff slightly different filesets, but not lineranges. Look, we already have both 'just functions' and dry-vs-complexity problem unsolved.
jonaf 10y ago
Static code analysis and a good IDE make this point considerably less important. For example, duplicating code is not nearly as expensive in my Spring (Java) application as it is in that shiny Nodejs webapp. It's still important, but solid tooling combats it somewhat.

seanhandley 10y ago

TL;DR Think before you type.

elcct 10y ago

High level of abstractions accumulate especially in big monolithic applications and I agree it takes time to get the feature in. Especially, you have to get an idea of what code has already been written and how it plays with the components before you start doing anything. First, to not repeat yourself and also to lessen the chances of doing something wrong. I think everything has its trade-offs and use cases. This approach in large applications could make you write less tests, as there is less amount of repeated code. On the other hand you can split your project into independent smaller parts and at the cost of sacrificing DRY you can get something more easy to understand, but then you have to put more effort on testing and integration testing to see if all bits play together correctly and as expected. I agree that the sense of what should be abstracted and what shouldn't comes with experience. It is also very important to get as much information about the problem - this helps choosing the right approach in the context of above.

brightball 10y ago
I tend to take DRY more from a standpoint of data and code to transform that data into other data. I'm less concerned about duplication in code that uses data than I am the code that determines whether the data itself is correct.

bpchaps 10y ago

My complaint with DRY - sometimes it's just significantly easier to learn a codebase or learn the problems that led to its structure/abstraction by just writing many of the underlying components yourself. Whether that code goes into the codebase is somewhat irrelevant from a learning perspective. On top of that, when 95% of the code you work with is based on a "Minimal Viable Product" (the bane of my existence), assumptions will probably be made incorrectly, or in a way that adds significant technical/organizational debt. By doing it yourself, the original assumptions can be put into a different light and can help reduce all sorts of debt in the future.

catnaroek 10y ago

The core of the problem is that people often confuse “parameterization” with “abstraction”: Parameterization is simply adding adjustable knobs. For example, a function can take as argument a gigantic list of flags, where each particular combination of flags makes the function do something slightly different. Abstraction is actually separating concerns. It's dividing a program in logically independent parts, whose correctness can be established individually, without worrying about the others. Abstraction means that software components only interact with each other through explicitly designated interfaces, often enforced by the programming language itself. Parameterizing is easy. Abstracting is difficult. Unfortunately, parameterization without abstraction is a source of headaches in the long run.

taeric 10y ago
There is a quote from Pólya that I love in margin notes of Concrete Mathematics. There are two kinds of generalizations. One is cheap and the other is valuable. It is easy to generalize by diluting a little idea with a big terminology. It is much more difficult to prepare a refined and condensed extract from several good ingredients. I think it applies here, fairly well. Just finding a way to not repeat yourself is relatively easy to do. Finding a way to not repeat yourself, but keep the same level of information conveyance is actually quite difficult.

sghiassy 10y ago

Thanks for the article. I agree with its main points

jstimpfle 10y ago

Another thing that is difficult to get right is choosing a suitable font...

jnordwick 10y ago

I am firmly in the DRY camp with a very important caveats: 1- Do no generalize so you can factor out like code too early. The first time just write it. The second time write it again even if only minimally different. The third time is a judgement call. By the fourth time you see the proper abstraction, something that might have been much different if you were to have refactored early on. Corollary to #1- Don't run ahead of yourself making all these placeholder functions, interfaces, and abstract classes with the expectation that you would have had to refactor this way later. Unless you done versions of your project a dozen times (even then, sometimes I don't even do it), you are most likely going to be wrong. 2- Don't refactor out single lines unless there a lot going on on that line or you have the experience to know (don't lie to yourself either) that this will be an important and growing line. 3- Sometimes performance can trump. And sometimes you do know this ahead of time, especially if performance is correctness as it is on some projects with very tight constraints. Besides that. I freaking hate seeing similar code everywhere, blowing up my instruction cache, making changes difficult. Please stop it.

InclinedPlane 10y ago
I disagree with your point 2 very strongly. A lot of people have a mental block when it comes to 1-line functions, and they really shouldn't, they can often be very helpful for a lot of reasons. Generally, the most useful tool in refactoring is to use extract method relentlessly, even into 1-line functions. More often than not, a well named function with well named parameters is one of the absolute best and most reliable ways to make your code as close to "self-documenting" as possible. But there are additional advantages as well. Spinning things off into function calls often makes refactoring easier. Because it discourages bad habits like local variable reuse (using variables as scratch pads) and excess mutable state. It also encourages a more functional style, which has a lot of benefits especially in multi-threading but also in the ability to reason about and determine the correctness of code. It can also make it a lot easier to see when you need to spin off functionality into other classes/modules. For example, if you have a member field and you have a ton of little one line snippets everywhere that anything happens with that field, that usually means you need to extract it out into a class and have those one-line snippets be various methods on that class. When all that code is inline, this can be harder to see. Additionally, there are lots of mental blocks that happen when you have inline "one liners". There's a bias to avoid "cluttering them up", even when it's necessary. For example, when you need to add error checking and error handling. When you've spun out the code into a method call, you tend not to have that block, you just add the error handling code because it's trivial, and going from a 1-line method to a 5-line method isn't a big deal. But bloating up another function by taking a sweet one-liner and replacing it with 5-lines of code can seem like clutter, so often times you just have an aversion to doing so. Additionally, sometimes those "sweet one-liners" can be more readable when they're broken up into multiple lines, and this is again something that is trivial and straightforward when it's spun out into its own method but will see resistance when the code is inline. This is true even if the code remains a single statement broken up into multiple lines to enhance readability. A lot of the time these patterns aren't obvious. Because the end result of spinning off one-liners is often not one-line method definitions, it's short methods that are a couple lines with several lines of extra error detection/handling. Or it's separate utility classes to encapsulate handling of special member fields. The idea of ensuring that you always have "meaty" functions is unhelpful, what's important is thinking about things from a perspective of abstraction, modularity, readability, and correctness.

leovonl 10y ago

The DRY principle issues usually apply very well to OO programming and the whole "framework" concepts, where individual libraries or pieces of the code cannot easily be reused for different projects. Another big issue in the framework case is the inversion of control, which further restricts how the reuse can happen and makes it very hard to reuse the code differently in an efficient way.

jimjimjim 10y ago

The idea of 'don't repeat yourself' is an ok guideline unless, like most proverbs/sayings/mottos/slogans it gets used as a absolute rule. feel free to ignore it, mock it and toss it aside if it leads to bad abstractions, highly convoluted structures or write-only code. anyone that has had to do maintenance or adding features to a large OO codebase from the early 2000s will have seen vastly massive class hierarchies where following the path of execution is a roller coaster ride through 10 files for what could have been a 30 line function. these days grep/vim/sublime/emacs/and the rest of the gang all have power search and replace editing functions. sometimes it's better for the whole world to just copy-paste-alter your code.

stcredzero 10y ago
an ok guideline unless, like most proverbs/sayings/mottos/slogans it gets used as a absolute rule. Amen! a roller coaster ride through 10 files for what could have been a 30 line function. sometimes it's better for the whole world to just copy-paste-alter your code. Then what happens when you have almost a hundred copy/pasted slightly rewritten 15-30 line variations on the same theme? How do you refactor then? (Yes, I have seen this is production systems, and yes, it was very critical code!) As you say, it comes down to cost/benefit. Basically, you just have to keep potential refactoring/rewrite costs down so that you are never trapped. Caveat: You can seldom predict the risks as well as you think you can. What you can depend on, is being observant to historical patterns in your codebase. It's hard to predict the business needs and the architecture in the future. On the other hand, it's often quite easy to see the historical trends in the codebase. Really, the analogy of the physical file room is a great one. (Or if you're not familiar, a library, a tool chest, or any kind of physical inventory system works too.) You can have a file clerk that seems like "super-filer" because he never "wastes" time by putting files back, but this never works out in the long term. The same goes for a filing staff that never reorganizes the file room. Also, one can generally see the long term disaster developing long before it results in the dramatic disaster. You can see which shelves are getting filled up, and which drawers are getting overstuffed, usually weeks or months ahead of time.
- Asooka 10y ago

sysk 10y ago

Am I the only one who thinks this sort of advice would be immensely more useful with some concrete examples?

Cyclone_ 10y ago
I was thinking the same thing, it's hard to judge this article without knowing exactly how much code duplication the author thinks is appropriate and in what circumstances.

stcredzero 10y ago

Yes, you read that right. Duplicating code (a.k.a copy + paste) can be a good thing. Namely when the abstraction that would replace repetitive portions of your codebase is a pain to understand. Yes, this 1000 times! DRY or any pattern/principle is not some sort of universal truth/declaration of ultimate goodness from your deity or some deep law like the Heisenburg Uncertainty Principle! It's just a rule of thumb. As a practical technologist, you must always implement with respect to the contextual cost/benefit. Taking any principle as an automatic ultimate good is simply intellectual laziness! I was in a company with a Java product and a Smalltalk product. The Smalltalk product was rather well factored. We had a policy of rewrites to keep things architectually clean. We had the fantastic Smalltalk Refactoring Browser as our IDE, and we weren't afraid to use it. (To this day, you can refactor an order of magnitude or more faster with it than you can in other languages.) The Java product consisted of many independent parts with tons of duplicated code between them. Definitely not DRY. (There were also 20X more Java programmers than Smalltalk, for a comparable product.) One thing I noticed, is that bugs on the Java side were more prevalent, but never affected the entire application. Whereas, the far better factored Smalltalk product could occasionally have the rug pulled out from underneath the entire application by the introduction of a library bug. (Though it would be very rapidly fixed.) Another thing at that shop: We had a policy of only empirically verified DRY. We never wrote code to DRY in anticipation: only in response to actual code duplication. Generally, actual practice and experience is better at producing exactly the needed architecture than the prognosticating cleverness of programmers. EDIT: Shower thought! (Yes, I literally got out of the shower just now.) There is a lot of emphatic but somewhat thoughtless application of principles and rules of thumb because it's actually motivated by signalling. The priority is actually on the opportunity to signal and not on the underlying cost/benefit calculation. There is a nice analogy for this in music. Once a musical scene gets a certain degree of prominence and/or commercial success, people start prioritizing sending the signal that they are part of that musical scene. So you will find bands and musicians using those techniques and stylistic flourishes, not because of the underlying aesthetic principles in support of the particular piece of music, but as an opportunity to signal. This happens in OO and also in the startup scene. (And, OMG, but is signalling given a lot of energy in the Bay Area!) So the insider move is this: Look for the underlying motivation behind the signal. Did this person jump on the opportunity to signal, or did they first seek out the information to make the determination of cost/benefit? How carefully did they do that? (Hopefully, we can get some of the hipsters/pretenders to at least go through the motion of doing the above. Even just going through those motions turns out to be beneficial.)

caseymarquis 10y ago

Article in a nutshell: Goodness = DRY×KISS

SideburnsOfDoom 10y ago

Yes, this is known as "Beware the Share" http://programmer.97things.oreilly.com/wiki/index.php/Beware_the_Share http://programmer.97things.oreilly.com/wiki/index.php/Beware...

RandomInteger4 10y ago

New Motto: Don't Repeat Yourself, Unless You Need To Repeat Yourself, In Which Case Repeat Yourself, But Be Careful How You Repeat Yourself ... DRYUYNTRYIWCRYBBCHYRY

zachrose 10y ago
Dry, un-try, wicker baby cherry.

gsmethells 10y ago

Figure out what varies => encapsulate that. Could be behavior, structure, or creation. This idea that you can avoid DRY in a positive way simply means one has yet to recognize that first statement.

sbov 10y ago

I've come to the conclusion that there are at least 3 kinds of DRY, and application varies: 1) There's a higher level abstraction in your code. Abstract at your own risk, unless you have at least 3-4 instances, or your architecture absolutely requires it. 2) There's wrappers around libraries and you only use it a certain way. I'm OK with this here. Sure, I could copy-paste the same exact parameters each time I use it in my code, but I will just write my own fiddleFoo() and use that every time. 3) There's when things MUST BE THE SAME. You should abide by DRY even if you only have 2 instances. E.g., we have an SOA and we DRY our routing. Routing MUST match or else things break.

fpoling 10y ago

DRY is rather language-dependent. In Java, where only single result can be returned from a function, it is often not worth the efforts to factor out common functionality if that requires adding an extra class just to pas few in/out parameters. In C++, on the other hand, the barrier to follow DRY is rather low as a I can literally move code to a helper function and easily parametrize it for reuse in another place.

toomim 10y ago

Here's a research article I wrote on this topic; there's a section on the "costs of abstraction": http://harmonia.cs.berkeley.edu/papers/toomim-linked-editing.pdf http://harmonia.cs.berkeley.edu/papers/toomim-linked-editing...

eternalban 10y ago
Interesting. Does it scale? (Skimmed the paper and believe have understood the process but examples you have are limited to n=2 clones. What if n>>2?)

wpietri 10y ago

I think this article could benefit from a few good examples. Sure, one should never apply principles in a rote fashion; they're guides to judgment. But the way we develop good judgment is through practice. Without seeing how he'd apply this in specific situations, I don't feel like I've really learned anything.

bogomipz 10y ago

I think the title should be "wrong abstractions." Wrongful means unfair/unlawful/unjust but it doesn't ever mean incorrect.

z3t4 10y ago

Something that I see more and more of lately is module exaggeration, like projects that have tens of thousands of small files that all depend on each other and are complexed together using modules. There are 5-10 imports then a few lines of code that binds them together and finally an export ... The worst part is that this is done by respectfully software companies and even taught in guides and tutorials.

breatheoften 10y ago

"It can be expensive to use the wrong abstractions, therefore you should copy and paste" is not sound reasoning. The _right_ abstractions can be incredibly (insanely, ridiculously) beneficial to a code base across all dimensions -- correctness, maintainability, performance. Searching for the right abstractions is definitely worthwhile - even if it sometimes incurs extra rewrite/refactor churn. The better the abstractions used, the easier the refactoring, the faster you find the good abstractions, the easier it is to change the code as your understanding of the problem grows ...

rdiddly 10y ago
He's saying duplication can sometimes beat inappropriate abstraction. You're saying appropriate abstraction beats both duplication and inappropriate abstraction. (Which I thought was obvious.)
dllthomas 10y ago
Sometimes what you have is a coincidence, not an abstraction. In that case, you should duplicate the coincidentally-identical implementation.

dllthomas 10y ago

DRY is a problem because it's so catchy and seems so self-explanatory. The original formulation of DRY spoke in terms of information. Any piece of information should exist in exactly one place in your code. As I've said before, modulo some caveats related to double-entry bookkeeping, this seems exactly right. An inveighing against anything that looks repetitive is not. The goal is not compression, it's ease of understanding and ease of making correct modifications. I've started using "Huffman coding" to refer to the practice of misapplying DRY by deduplicating superficial similarity rather than information content.

zachrose 10y ago
This is a cogent way to frame the acronym. There are times when repeating yourself is just accidental. I might have the same middle name as a coworker. Do we actually "share" that middle name? That depends on who's asking, or what system you're trying to build. But probably not.

Rygu 10y ago

For a more visual explanation of the "cost of abstractions" watch this presentation by Cheng Lou (Facebook). https://www.youtube.com/watch?v=mVVNJKv9esE https://www.youtube.com/watch?v=mVVNJKv9esE

InclinedPlane 10y ago
This is an excellent talk which introduces some very solid concepts and breaks down the problem in a very sophisticated way. I hope more people watch this. In fact, I think this talk is superior to the linked article. You should consider submitting this talk directly to HN.

cle 10y ago

My own experience with DRY is that it has diminishing value in a distributed systems ecosystem. There is definitely still value, but code coupling between components of a distributed system is very costly. I'm pretty fed up with coordinated deployments, big bang upgrades, etc., so I'm very conservative about applying DRY. Often I'd rather copy a bit of code to avoid the dependencies, which usually pays off better in the long run.

smnplk 10y ago

I think we need to address this giant elephant in the room called object oriented programming. https://www.youtube.com/watch?v=IRTfhkiAqPw https://www.youtube.com/watch?v=IRTfhkiAqPw

z3t4 10y ago

DRY etc should just be guidelines, not rules. I hate when something is made overly complex just to avoid repeating, global variables, long functions or large files. For example complicated one liners in a imperative language.

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]

pm24601 10y ago

Easily solved: When writing an abstraction, document with a date the reason for the abstraction. Seriously, a comment that goes like this: "2009 sept 20 : Peter Pan: this abstraction is to hid the ugliness of using SQL Server 2003 which is still in use by Slow Corp." ... now in 2016 a maintainer knows the assumptions and the reasons for the abstractions and can make a much clearer decision about how the abstraction should be treated.

coldtea 10y ago

Off topic, but since somebody will always complaint about the bad layout or the difficulty of reading of a blog post submitted to HN, I'd like to point out the excellent readability of this one.

firloop 10y ago
The stark blacks kind of burned my eyes, but I definitely agree it was readable.
- coldtea 10y ago
  One can always lower the brightness of their screen, but they can't easily change the contrast in the font/bg. Besides, research has shown that plain ole black on white is the best combo for readability.

jayvanguard 10y ago

This article needs to be shown to all upcoming architects, senior developers, and design minded engineers. Too often the balance between abstraction and specialization isn't thought about explicitly but rather dogmatic rules kick in.

agentultra 10y ago

All too often the kind of "abstractions" we talk about in programming are the leaky, code-indirection kind. These abstractions tend to proliferate in a single code-base over generations of programmers. It leaves one wondering if these thousands of lines of code actually compute anything. In my experience the software at an organization is often a reflection of its culture. Thick with leaky abstractions, interfaces, facades, and indirection? Years of frustrated mediocre programmers, each slightly confused by the previous generations' shenanigans, led by a clueless management team that care more about deadlines than deliverables. There are signs that people are trying to do better but the code takes years to show improvement as it resists change at every corner. A crystalline entity designed and driven by data, never doing more work than is required? A team that has had little churn and is led by passionate engineers often more concerned with deliverables -- at least at first, because it's often fairly easy to make changes when your code-base adheres to a well-defined algebra, is thoroughly tested, and even has a well-formed specification of its critical components. DRY is as good an idiom as "measure twice, cut once." The difficulty is that software is often more complex to understand than a cabinet. However I can't slight it for being a bad idiom because of the wide variability in skill for such a difficult task as programming. We need something to teach new programmers that at least get them thinking about the abstractions they wreak havoc with.

thr0waway1239 10y ago
Your comment reminded me of Edmond Lau's book - The Effective Engineer [1] - where he talks about putting in a good amount of effort into the onboarding process for new engineers. His premise - having a senior engineer spend an hour a day for the first month helping the new employee with explaining the existing abstractions being used, the underlying design of various systems, etc. - would still be only about 20 hours, which is still only 1% of the number of hours that employee will spend in their first year - about 2000 hours. As a result, I believe that armed with that knowledge, the new employee is likely to be much more productive, failing which, at least cause less damage to the code base. I would say that the first example you mention - leaky abstractions et. al. - are just as much (or maybe more) due to poor onboarding as they are due to the frustration of mediocre programmers. There is a lot to be said for good process, which software engineering as a discipline falls short of quite consistently. [1] https://www.amazon.com/Effective-Engineer-Engineering-Disproportionate-Meaningful/dp/0996128107/ https://www.amazon.com/Effective-Engineer-Engineering-Dispro...

kazinator 10y ago

Any time you repeat some block of code twice, you should religiously turn it into a function which is called twice with the same arguments. Any time you call a function twice with the same arguments, you should religiously write a two-iteration loop around just one call. Any time you have more than one two-iteration loop in the same program, you should write a 'dotwice' macro and use that instead. Any time you have a function call sitting in a loop just for the sake of being repeated, you should avoid the argument expressions being evaluated multiple times; you must religiously evaluate the arguments to temporary variables outside of the loop, and in the looped function call, refer only to the temporaries. In order not to repeat this pattern itself you need a proper "calltwice" macro, and to be working in Lisp, ideally: (calltwice yourfunc expr ...). yourfunc and each expr are evaluated once, then the resulting values are applied to the function twice. This is a DRY town, damn it!

lambdabit 10y ago

Can we just ban coding posts that have no code? They usually give you an illusion you learned something but are a total waste of time. It's like reading a self-help post on following your biggest dreams or something. The hard part isn't these big beautiful ideas, it's how to actually put it in practice. How to actually take something away and measure whether abstractions or duplications are more costly in real code??

threepipeproblm 10y ago
This seems like an anti-intellectual argument. For example would "ban" many of Dijkstra's famous numbered papers. I'm guessing you haven't read them. While we're at it, why don't "we" just "ban" - mathematical proofs without numbers - all criticism, as it's about art but contains no art - all papers in the field of music theory, since they aren't music.
- lambdabit 10y ago
  It's definitely anti-intellectual. But people who run this site themselves say they're interested in high signal to noise ratio. I don't think those are good comparisons. Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example.
  - threepipeproblm 10y ago
    >> Better example would be a post about how to write rigorous math proofs and then not giving a rigorous proof as an example. Oh, you mean like some of Dijkstra's numbered papers? https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1012.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E... I think "examples might help" is valid feedback, but the author has sort of provided a reason why examples aren't included: in small projects the abstractions are usually worth it, was my read. So while the author could talk about examples, he probably couldn't include then in a piece of this size. "Can we ban" is what has rubbed me wrong way. Who is "we". How are "we" "banning"? Does this just mean you don't agree with this post being highly upvoted? Or that you want to circumvent the opinions of those who upvoted with a "ban"? It is interesting also to think about the irony of an argument that one should work in terms of concrete examples of the subject matter, in objection to a piece that argues one should not always abstract things.

pyronik 10y ago

Sometimes I think programmers and theologians have a lot in common. Both have dogmas and articles of faith which are backed by internally consistent logic. Now to the comments to hear the Catholics and Protestants debate!

kefka_p 10y ago

Frankly without encapsulations inside of encapsulations, the author would have never been able to make this blog post. Reference TCP packets wrapped inside of Ethernet 2 frames as the basis of the entirety of the internet. I don't think this is as "extreme" as some perceive it.

buzzybee 10y ago

I was younger and I thought that the right abstraction - and more specifically, the right language feature - would surely lead me to higher productivity. By invoking a certain keyword my code would magically be better in some way. However, language features are not the only abstractions you have available. In fact, they're typically the least interesting ones you could apply to a problem - any yokel can lift code out and slap parameters, annotations, reflection, or generic type signatures on it. But to reach interesting abstractions, you have to be patient enough to do discover the shape and flow of the code over time and subsequently apply exactly the right data structures and algorithms to either reproduce the same shapes in an abridged form, or different shapes that achieve the goal more efficiently. So when I position myself as anti-DRY, which I often do now, I'm saying, don't succumb to the thinking that just shovelling things around and slightly repackaging them will add up to what you wanted. An actually useful abstraction cuts so deep that it may amount to a different thing altogether. The language features are there to solve very straightforward situations that are known to reoccur time and again. Do not use them in clever ways.