14 ms·

The Wrong Abstraction (2016)

654 points by mkchoi212 6y ago

sheeshkebab 6y ago
I’m not sure why this is #1... but since it is, both of these - duplication and wrong abstractions - are otherwise known as technical debt.
- dasil003 6y ago
  Not necessarily. Technical debt is when you do something quick and dirty to get a feature out in the short-term knowing that it won't be maintainable, scalable, etc, but you do it anyway with the expectation that you'll fix it later. Some duplication and wrong abstractions are caused by this, but definitely not all.
  - hrhrhrd 6y ago
    No, technical debt is a very general category that includes deliberate hacks, structural flaws, and small mistake bugs. It's anything that over time will damage the code base, duplications and wrong abstractions being very much included in that
    - dasil003 6y ago
      You're welcome to your own definitions, but personally I keep bitrot, deferred maintenance, and "structural flaws" (which can be subjective and dependent on use cases and scale) out of the bucket of technical debt since it robs the metaphor of a defining aspect: intentionality. Debt is not something that happens passively as the world changes around you, it's something which you sign up for.
      - quinnirill 6y ago
        If you unintentionally destroy property and have to pay for it, you’re in debt. We even have a concept of life debt. Some debt is intentional, some incidental. Most technical debt I’ve seen was not intentional, just a well meaning design that was created to serve a purpose that eventually outgrew it, and that’s when the interest started to pile up. And happening passively is exactly what it does, interest rates change, your ability to make downpayments change. All part of the very well functioning metaphor in this context.
twirlock 6y ago
cc my know-it-all coworker
chiefalchemist 6y ago
Why not simply duplicate the abstraction, refactor as needed, and adjust the necessary caller(s)? Having to know, find and maintain the individual duplications feels dirty and its own way wrong. Choose your wrongs wisely?
Pxtl 6y ago
Every Line Of Business codebase I've worked on has been the worst "there I fixed it" copypasta spaghetti, and has never made it to the point where "maybe we shouldn't add a parameter to this existing, cleanly abstracted method to handle this new similar-but-distinct use-case" was anywhere near my radar for abstraction. I would love to have developers where my problem was "maybe you piggybacked on existing code too much, in this case you should've split out your own function".
- mlthoughts2018 6y ago
  It’s been the exact opposite for me. The spaghetti code has always come from poorly conceived abstractions and the massive problem of inverting an API to reimplement functionality through the API that should be extensible within the API (but fails to be because of poor choices in abstraction or abstracting prematurely). Later on that spaghetti code gets labeled as lacking abstraction, similar to what you are saying, despite the actual problem being too much abstraction and poorly designed abstraction that became load bearing in a way where everyone decides that living with API inversion as a reality is the lesser evil and figures they’ll probably quit the company and move on to greener pastures before it becomes their headache to deal with. https://en.m.wikipedia.org/wiki/Abstraction_inversion https://en.m.wikipedia.org/wiki/Abstraction_inversion
  - hesdeadjim 6y ago
    Absolutely this. I’d rather look at 200 lines of linear, inline documented code then a spaghetti mess of “helper” functions that do nothing better than obfuscate everything going on. I’ve had a strict rule with my team of “1, 2, N”. I don’t want to see an abstraction until we’ve solved a problem similarly at least two times, and even then an abstraction may still be a poor idea. Abstraction is an especially poor idea early in a project because often you only half know what you’re making (I’m in games). Requirements change, or a special case needs to be added, and all of a sudden you are trying to jam new behavior into “generic” helpers without breaking the house of cards built around them.
    - hackinthebochs 6y ago
goto11 6y ago
I'm skeptical because it is really easy to un-share code by copying it into multiple places but it is very hard to unify duplicated code. So I prefer to err on the side of sharing. But yes, you should be ready to change sharing into duplication if you realize the code is just "accidentally similar" and need to evolve in separate directions. In practice I have seen a lot more pain due to duplicate code compared to the issue of over-abstracting code, because the latter is much easier to fix.
- joeframbach 6y ago
  On the other hand, it's really difficult to know who is using that shared code. If you make an innocuous change in a shared method, it could affect someone else you don't know.
  - TheCoelacanth 6y ago
    It's very easy with proper tooling.
  - BoiledCabbage 6y ago
    Outside of publishing a public API almost any modern language and enviroment should make this easy.
  - bcrosby95 6y ago
    It's a million times easier than figuring out if those minor differences in duplicate code are accidental or on purpose. As bad as a flag-laden method might be, you know the intent of all callers.
  - mcintyre1994 6y ago
    I find it much easier to find the call sites for a function than to find code that’s duplicating or a variant of the code I just fixed a bug in so we can figure out if the same bug is latent in the duplicates too.
  - dtech 6y ago
    Not in any modern language or IDE. Not to mention that would indicate a hole in the test suite
gorgoiler 6y ago
Brilliant insight. Always remember: (1) make it work, (2) make it right, (3) make it fast. 80% of projects get scrapped in between (1) and (2) because you end up realizing you wanted something completely different anyway.
- nicoburns 6y ago
  On my projects code doesn't make it into the main branch until it gets to at least (2).
- willcipriano 6y ago
  > (1) make it work, (2) make it right, (3) make it fast. I've always disagreed with this. In my view you should make it a habit to write optimized code. This isn't agonizing over minor implementation details but keeping in mind the time complexity of whatever you are writing and working towards a optimal solution from the start. You should know what abstractions in your language are expensive and avoid them. You should know roughly the purpose of a database table you create and add the indexes that make sense even if you don't intend to use them right away. You should know that thousands of method lookups in a tight loop will be slow. You should have a feel for "this is a problem someone else probably solved, is there a optimal implementation I can find somewhere?". You should know when you use a value often and cache it to start with. Over time the gap between writing unoptimized and mostly optimized code gets smaller and smaller just like practice improves any skill.
  - criddell 6y ago
    > You should know that thousands of method lookups in a tight loop will be slow. That's not always the case. Modern compilers do a lot of things like inlining and unrolling. These days I mostly try to write code that is easy to understand.
    - willcipriano 6y ago
      > Modern compilers do a lot of things like inlining and unrolling Smart ones do, I've been writing Java lately and that behavior tends to be unpredictable and rare[0]. I'd use a inline keyword if I had one, or preprocessor directive of some kind if I had that but I don't. I agree it's harder to read but I feel like changing a JVM flag to get a behavior that I want is more inscrutable than having a long method with a comment noting that this in inlined for performance reasons. With modern machines and the price of memory I tend to lean hard to the memory side of the time-memory tradeoff. [0]"First, it uses counters to keep track of how many times we invoke the method. When the method is called more than a specific number of times, it becomes “hot”. This threshold is set to 10,000 by default, but we can configure it via the JVM flag during Java startup. We definitely don't want to inline everything since it would be time-consuming and would produce a huge bytecode." https://www.baeldung.com/jvm-method-inlining https://www.baeldung.com/jvm-method-inlining
ridaj 6y ago
Previously discussed here: https://news.ycombinator.com/item?id=17578714 https://news.ycombinator.com/item?id=17578714
- arendtio 6y ago
  I find that first comment particularly insightful. However, I am not sure about the order of state and coupling. To me it seems to depend on the language, as for functional languages, avoiding state is king and in object oriented environments, coupling could be a more important factor.
kolinko 6y ago
I wish this article was available two years ago when I tried to explain this to a bunch of juniors working for me...
- nnutter 6y ago
  “ Posted on January 20, 2016 by Sandi Metz.”
  - kolinko 6y ago
    Damn, I wish I saw it back then :)
brandonmenc 6y ago
Junior programmers duplicate everything. Intermediate programmers try to abstract away absolutely every line that occurs more than once. Expert programmers know when to abstract and when to just let it be and duplicate.
- leafboi 6y ago
  The master never duplicates and all his abstractions are intuitive, readable and flexible.
kevsim 6y ago
Relevant post from earlier today https://news.ycombinator.com/item?id=23735991 https://news.ycombinator.com/item?id=23735991
klyrs 6y ago
I use DRY in two ways. The first is that I'm happy to make 2 or 3 copies of a snippet before promoting that to a new function. The second is when I find a bug in a duplicated snippet. I'll mend the snippet and its duplicates, once or twice before promoting it to a function. In the rarer (in my line of work) instance that a common snippet gets used with several intrusive variations, I usually document the pattern. It's tempting to use templates, lambda functions, closures, coroutines, etc but far simpler to duplicate the code. But again, if a bug (or refactor) crops up and I need to fix it in many places, then I'll spend some time thinking about abstraction and weigh the options with the benefit of hindsight.
jpswade 6y ago
You can’t plan for what you don’t know. This is why I like the "Rule of three"[1]. Only once you've done it three times will you truly begin to understand what the abstraction might need to look like. 1. https://wade.be/2019/12/10/rule-of-three.html https://wade.be/2019/12/10/rule-of-three.html
- geophile 6y ago
  Exactly. With experience, you learn not to abstract too soon.
  - hesdeadjim 6y ago
    Seems so counterintuitive, but it’s one of the most important lessons I’ve learned in 15 years of development experience.
- deleted 6y ago
  [deleted]
- ed312 6y ago
  Any advice on teaching this to junior engineers? Seems like folks with 3-5 years of experience keep trying to not only over-abstract but also keep re-inventing the wheel with abstractions (vs looking for existing libraries).
  - deleted 6y ago
    [deleted]
  - leafboi 6y ago
    It's largely because they're dealing with an area with no theoretical tools. Any time you hit an area that are full of people "Designing" solutions/abstractions rather then "Calculating" an optimal solution/abstraction you know you've hit an area where there's very little theoretical knowledge and most people are just sort of wandering chaotically in circles trying to find an "optimal" solution/abstraction without even a formal definition of what "optimal" is.... I mean what is the exact definition of the "perfect abstraction"? What is bad about duplication what is a bad over abstraction and what is this "cheaper" cost that the title is talking about? It's all a bunch of words with fuzzy meanings injected with peoples biased opinions. That being said theories on abstractions do exist. If you learn it you'll be at the top of your game; but it's really really hard to master. If you do master it, you'll be part of a select group of unrecognized elites in a world of programmers that largely turn to "design" while eschewing theory. Here are two resources to get you started: The Algebra of Programming: https://themattchan.com/docs/algprog.pdf https://themattchan.com/docs/algprog.pdf Program Design by Calculation: http://www4.di.uminho.pt/~jno/ps/pdbc.pdf http://www4.di.uminho.pt/~jno/ps/pdbc.pdf You will note that both of these resources talk about functional programming at its core which should indicate to you that the path to the most optimal abstraction lies with the functional style.
seanalltogether 6y ago
This quote from John Carmack speaks very succinctly to the problems that many abstractions in a code base can cause, and it's a constant reminder for me when building out business logic. > "A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in." https://www.gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php https://www.gamasutra.com/view/news/169296/Indepth_Functiona...
- hackinthebochs 6y ago
  But abstractions reduce possible state and allows you to specify that state in obvious ways, e.g. on function parameters. Do not underestimate the power of functional boundaries.
  - ben509 6y ago
    They also tend to impose a degree of discipline. I've often found myself wanting to shove a parameter in somewhere and realized I didn't need the damned thing.
- hesdeadjim 6y ago
  This is one reason I love working in the Unity ECS framework. Your data is public and state can’t hide. Your systems are still free to contain a plethora of bugs, but they are easier to track down due to the functional nature of a system. In the regular Unity OOP land, developers inevitably sprinkle state everywhere. Coroutines are by far one of the worst offenders. Good luck seeing the current executing state of your game when it’s hidden in local variables inside a persistent function body...
- jonnycat 6y ago
  Reading that article and the context of the quote, it appears that Carmack is using that statement to extol the benefits of functional programming styles, not commenting on abstraction.
  - seanalltogether 6y ago
pps43 6y ago
Related to http://yosefk.com/blog/redundancy-vs-dependencies-which-is-worse.html http://yosefk.com/blog/redundancy-vs-dependencies-which-is-w...
deleted 6y ago
[deleted]
crazygringo 6y ago
Another tip is: if you're duplicating, and they're not lines of code that are visually obviously next to each other, then leave a comment next to both instances mentioning the existence of the other. There's nothing inherently wrong with duplication, except that if you change or fix a bug in one, you need to not forget about the other. Creating a single function solves this... but at the potential cost of creating the wrong abstraction. When you're at only 1 or 2 extra instances of the code, just maintaining a "pointer" to the other case(s) with a comment serves the same purpose. (Of course, this requires discipline to always include the comments, and to always follow them when making a change.)
- stormdennis 6y ago
  Would the risk forgetting to update the comments not be a reason for creating a wrapper method that handled calls to both and contained the relevant advice?
deleted 6y ago
[deleted]
djhaskin987 6y ago
Mods this article is old, should be labeled 2016.
gumby 6y ago
Early de duplication is the equivalent of early optimization: a bad idea that boxes you in. Duplicate code is a sign that there could be a generalization missing.
tarkin2 6y ago
"With C you can shoot your own foot. With C++ you can blow your own leg off". I feel the same is true here. The abstraction may be right at the time of writing, yet further on it often becomes not only wrong, but a massive hindrance. With time and effort, hacky code and be worked into shape. An eventual wrong abstraction normally means a rewrite.
random3 6y ago
This is so true, but so shallow too. I think the big mistake is to treat the code as "the main thing" when in reality it's just a model (a golem) mimicking some "other thing" We're missing an entire set of code characterizations. Yes we have a "pattern language" but there's not much to characterize it structurally wrt "code distance" from one part of the code to the other (e.g. in call stack depth as well as in breadth). And again all of this needs to happen wrt the "abstraction" not the code itself. Having 10 methods 90% duplicated in a single file with 10% pecent difference is many times better than trying to abstract it. Having the same "unit conversion" function duplicated in 3 parts of the code can be disastrous. These two examples are very easy to see and understand, but in reality you're always in a continuous state in between. And "code smells" like passing too many parameters or doing "blast radius" for certain code changes are only watching for side-effects of a missing "code theory". An interesting book on the topic is "Your code as a crime scene". The bottom line is we're trying to fix these problems over and over again without having a good understanding of what the real problem is and this leads to too many rules too easy to misinterpret unless you are already a "senior artist"
- ijidak 6y ago
  > Having the same "unit conversion" function duplicated in 3 parts of the code can be disastrous. This. I feel like it's really about cognitive load to remember and recognize the differences. Duplication in 3 distant files, places a heavy load on the developer to: 1. Discover the duplication 2. To grasp the reason for the differences in the 3 different locations. 3. Remember these things Whereas when the duplication is in the SAME file, #1, #2, and #3 can become very manageable cognitively. Now the question changes to.. Is the cognitive load of dealing with the different special cases in a single de-duplicated method GREATER than simply leaving them in separate methods? Often the answer is duplication WITHIN a file is less of a cognitive load. Whereas duplication ACROSS files is a heavy cognitive load. Minimizing cognitive load minimizes mistakes. And minimizes developer fatigue. Thus boosting productivity. At least, that's my development philosophy, even though I've never seen it in a design pattern or a book. It just seems to make sense.
bcrosby95 6y ago
I find it interesting that comments on these articles mainly discuss 1 aspect about it. But rarely this part: > Don't get trapped by the sunk cost fallacy. In my experience, yes, programmers are hesitant to throw out an abstraction. Why not work to change this, rather than telling people not to abstract?
- ben509 6y ago
  I don't think it's a sunk cost fallacy. I think the hesitation is more for social reasons, often not wanting to do a big pull request that's going to be scrutinized.
  - Tainnor 6y ago
    "Big pull requests" that are unannounced are always problematic because who wants to be the person saying "all of this work you've done is wrong"? In such situations, it's good to get buy-in from other people before attempting to do such a thing. Make a proposal for a big change and discuss it. There's still a chance that, in the implementation it doesn't work as nicely as believed initially, but at least now it's less likely that the idea will be rejected wholesale during code review.
naringas 6y ago
sometimes it's better to copy and paste some code only to make each copy diverge more and more over time (somewhat like a starting template) as opposed to introduction an abstraction to generalize some slightly different behaviors only to use said abstraction twice. this makes even more sense when the code will live on in different programs there's a point when incurring the cognitive overhead costs of the abstraction become worthwhile, probably after the 3rd time. but my point is that it's also important to consider that the abstraction introduces some coupling between the parts of the code.
- rightbyte 6y ago
  I find it easier to read long functions of code than jumping around in helper functions or abstractions. Especially if I am not familiar with the code base and don't know common functions by heart.
adamkl 6y ago
Sandi mentions this during a talk she gave on refactoring a few years ago. [0] It’s a great little video for showing junior developers how a messy bit of code can be cleaned up with a few well chosen OOP patterns (and a set of unit tests to cover your ass). [0] https://youtu.be/8bZh5LMaSmE https://youtu.be/8bZh5LMaSmE
- voodoologic 6y ago
  I'm very partial to this talk about proper abstraction (and not just for junior developers): https://www.youtube.com/watch?v=OMPfEXIlTVE https://www.youtube.com/watch?v=OMPfEXIlTVE
ulisesrmzroche 6y ago
“Premature optimization is the root of all evil”
Xlurker 6y ago
I'd rather ctrl-f and change code in multiple places than deal with abstraction hell.
pierrebai 6y ago
Counter: Refactoring is far, far, far cheaper than duplication or wrong abstraction. Duplication means you lose the wisdom that was gained when the abstraction was written. It means that any bug or weird cases will now only be fixed in one place and stay incorrect for all the places you duplicated the code. About the rule of three: I personally extract functions for single-use cases all the time. The goal is to make the caller be as close to pseudo-code as possible. Then if a slightly different case comes up, I will write the slightly different case as another function right next to the original one. Otherwise, the fact that you have multiple similar cases will be lost.
- fiddlerwoaroof 6y ago
  Yeah, the rule of three is misleading: having a name for three lines of code that do “one thing” is almost always a win and nothing prevents a future developer from either inlining that function, if it was a bad idea, or duplicating and modifying the function.
- twirlock 6y ago
  Yes, everyone on the planet understands why duplication is not ideal. Literally nobody entered the discussion not knowing that. You clearly don't grasp what abstraction is fyi.
- jonahx 6y ago
  Counter-counter: Refactoring is by far the most expensive and error prone activity in programming. It can also be one of the most valuable. But unless it's trivial, it's the most mentally arduous and time-consuming work you do as a programmer.
  - titzer 6y ago
    I disagree. I think debugging is the most expensive activity a programmer ever does. Refactoring is a luxury that you have when you don't have bugs or time pressure to ship/fix something. Debugging potentially requires you to load the entire context of the (incorrect) program into your head, including irrelevant parts, as you grope around to figure out a.) what actually went wrong, b.) why it went wrong, and c.) how to modify the existing system in a way that doesn't make it worse. Debugging is reverse-engineering under the gun. It has huge cognitive load. Especially debugging a production system with a difficult to reproduce bug in a deep dark part of the code. It's a nightmare scenario. Refactoring, on the other hand, often happens with incomplete knowledge, and can be quite local. I've seen zillions of refactorings that are done with incomplete knowledge that are local improvements (and many that were not global improvements).
Tainnor 6y ago
I feel some people here are misunderstanding the blog post. Sandi Metz IMHO doesn't claim that the problem occurs at step 2 or 3. She doesn't claim that it's wrong to introduce abstraction when there is duplication. What she is saying instead is that the problem occurs from step 6 onwards: when you find yourself wanting to reuse an abstraction that, regardless of whether it made sense in the first place or not, has outlived its usefulness. I think this is in agreement with other points that she often makes, about being bold, but methodical about refactorings. The whole discussion about "you should never abstract away code before you see the third duplication" has little to do with the article, and I'm also really not sure it's good advice.
- kwhitefoot 6y ago
  > Sandi Metz IMHO doesn't claim that the problem occurs at step 2 or 3. But the headline does. I had to read quite a long way down the page to discover that all she is advocating is what i have always done: deduplicate when practical, undo the duplication when new requirements make it incorrect and push the unique parts into the callers.
  - DougBTX 6y ago
    > But the headline does. That’s not really fair, it repeatedly says “wrong abstraction”, in the title and in the article. At steps 2 and 3 it is still the right abstraction, duplication only becomes better when it is the wrong abstraction.
- tarsinge 6y ago
  That’s not what I get from the article. The problem does indeed occurs at step 2 and 3: leave duplication alone and don’t introduce abstraction if you are not sure about future requirements.
  - Chris_Newton 6y ago
    Taken to its logical conclusion, doesn’t that argument mean we would almost never introduce any abstractions at all? That doesn’t seem very practical compared to the alternative of introducing abstractions if they are useful at the time but remaining willing to change or remove them again later if the situation changes.
hackinthebochs 6y ago
I couldn't disagree more. There is no such thing as abstracting too early (this does not go for structural abstractions like factories, singletons, etc). The best code is code you don't have to read because of strong, well-named functional boundaries.
amelius 6y ago
A manager once asked me: please reuse as much code as you possibly can. This reminded me of that.
deleted 6y ago
[deleted]
hota_mazi 6y ago
> prefer duplication over the wrong abstraction Such a strange advice. If you're able to recognize the wrong abstraction right away, surely you would not use it, right?
- mrkeen 6y ago
  The reason is that you won't know it's the wrong abstraction until it's time to modify it or add new features.
- allenu 6y ago
  I think the intent was to communicate that abstractions aren't always right. Some people might think that because there's duplicate code and that the abstracted code maps to the duplicated code 1 to 1 and leads to fewer lines in total, it's a good abstraction, not realizing that there are costs to doing this that may not be aware of.
jack_h 6y ago
I would say that if developers are hacking on an abstraction that is ill-suited to the task until the code base is a nightmare, they will take this advice and duplicate code until it's a nightmare. The fact of the matter is every line of code that is written has an associated cost. Developers all too often pay that cost by incurring technical debt.
bob1029 6y ago
This whole thing exists on a normalized/de-normalized spectrum. The problem is that both ends have pros/cons. On the normalized side, you have the benefit of single-point-of-touch and enforcement of a standard implementation. This can make code maintenance easier if used in the correct places. It can make code maintenance a living nightmare if you try to normalize too many contexts into one method. If you find yourself 10 layers deep in a conditional statement trying to determine specific context, you may be better off with some degree of de-normalization (duplication). On the de-normalized side, you have the benefit of specific, scoped implementations. Models and logic pertain more specifically to a particular domain or function. This can make reasoning with complex logic much easier as you are able to deal with specific business processes in isolation. You will likely see fewer conditionals in de-normalized codesites. Obvious downsides are that if you need to fix a bug with some piece of logic and 100 different features implement that separately, you can wind up with a nasty code maintenance session. I find that a careful combination of both of these ideas results in the most ideal application. Stateless common code abstractions which cross-cut stateful, feature-specific code abstractions seems to be the Goldilocks for our most complicated software.
why-el 6y ago
Rob Pike discusses similar points in this section of his talk on Go Proverbs https://www.youtube.com/watch?v=PAAkCSZUG1c&t=9m28s https://www.youtube.com/watch?v=PAAkCSZUG1c&t=9m28s.
dfischer 6y ago
Reminds me of this discussion: https://news.ycombinator.com/item?id=12120752 https://news.ycombinator.com/item?id=12120752 (John Carmack on inlined code).
jbmsf 6y ago
One of the reasons duplication is used badly is that it is one of the easiest abstractions to recognize. One of the ways I've seen DRY go horribly wrong involves reusable code units evolving into shared dependencies that often interdepend in complex ways. Unfortunately, the problems of such a system are observed much later than the original code duplication and fewer people have the experience to see it coming.
leto_ii 6y ago
As I gain more and more experience (I would now call myself more or less a mid-level developer), I find that the distinction that matters is not abstraction vs duplication, but the one between developer mindsets. I have many times met/worked with people who think the main task of the developer is to 'get shit done'. Regardless of their level of experience, these developers will churn out code and close tickets quite fast, with very little regard for abstraction, design, code reuse etc. Conversely, the approach that I feel more and more is the correct one is to treat development as primarily a mental task. Something that you first think about for a while and try to design a little. The actual typing will in this case be a secondary activity. Of course, this doesn't mean you shouldn't iterate on your design if during execution problems come up. Just that the 'thinking' part should come before the 'doing'. My feeling is that with this second approach the abstraction/duplication trade-off will not matter so much anymore. With enough experience you will figure out what you can duplicate and what you can design. And when you design you will develop an understanding of how far you should go. Approaching development as a task of simple execution I think inevitably leads to illegible spaghetti down the line.
- Tainnor 6y ago
  I agree that many issues with bad code could really be avoided by first thinking about the solution a bit, of which the code is just an expression. I'm not advocating weeks of architecture astronauting without code feedback - because practical considerations (e.g. the compiler can't deal with this kind of code due to some limitations) matter - but some people seem overeager to just start writing some code "and see what happens".
scrozart 6y ago
DRY gets abused regularly in my experience. It doesn't stop at method/class abstractions either; I've seen entire microservices & plugins developed to ensure each app doesn't have that one chunk of auth code, for instance, even though they each may have subtly different requirements (those extra params again). The logical end to this sort of thing is infinitely flexible/generic multipurpose code, when the solution is really, probably increased specificity. DRY is probably the lowest-hanging fruit for practices/patterns, and I think this leads to a disproportionate focus on it.
- hesdeadjim 6y ago
  It’s also easy compared to solving new problems, so it can be an emotionally safe way of feeling productive. Failure is difficult to measure until the abstraction falls flat on its face months later, at which point it can be chalked up to the demons of “changing requirements”.
  - zbentley 6y ago
    That is a very, very important point; well put. The "of course it sucks: changing requirements!" boogeyman means one of two things: "the code was written to do the wrong thing because requirements changed/weren't communicated" or "the code was hard to change when it needed to do a new thing". Figuring out which of those two is in play is very important.
preommr 6y ago
I strongly dislike this article because the title is much broader than most of the substance of the article. Advising not to overextend an abstraction is inarguable. The actual title "Duplication is far cheaper than the wrong abstraction", and the thing that people will really discuss, is a loaded statement that's going to need a lot of caveats.
- deleted 6y ago
  [deleted]
zarathustreal 6y ago
I’ve seen this “hot take” a few times before and even see developers that I would have considered very good agree with it. Consider that all code is computation, this is the point of a computer: to compute. Consider that abstraction doesn’t seem valuable -to you- for a multitude of reasons. Perhaps you’re using a flawed paradigm that emphasizes objects over computation. This would obviously mean abstraction -increases- the difficulty of reasoning about your code. Perhaps you don’t have a mental map of appropriate abstractions due to a lack of education or knowledge gap, this could lead you down the path of creating abstractions which reduce duplicate characters or lines of text but are not logically sound (“leaky abstractions.”) All of these things come together in a modern “enterprise” software environment in just the right way such that abstraction starts to seem like a bad idea. Do not fall into this line of thinking. Study functional programming. Study algebraic structures. Eventually the computer science will start to make sense.
layer8 6y ago
The main takeaway from the article is that abstractions which have become inadequate should be corrected (removed and/or replaced by adequate ones) as soon as possible. A corollary is that abstractions should be designed such that they can be replaced or removed without too much difficulty. A common problem in legacy code bases is not just that they contain many inadequate abstractions, but that the abstractions are entangled with each other such that changing one requires changing a dozen others. You start pulling at one end and eventually realize that it’s all one large Gordian knot. One thing that I learned the hard way over the years is to design abstractions as loosely coupled and as independent from each other as possible. Then it becomes more practical to replace them when needed.
adrianmonk 6y ago
Two questions (genuine, not rhetorical): (1) How much of this is because it's actually hard to back out of the wrong abstraction and pivot to the correct one, and how much of it is other causes? The article hints at this with, "Programmer B feels honor-bound to retain the existing abstraction." Why do they feel this way, and is the feeling legitimate? Do they lack the deep understanding to make the change, or are they not rewarded for it, or are they unwilling to take ownership, or is it some other reason? I could see it going either way, but the point is to understand whether you're really stuck with that abstraction or not. (2) How much of the wrong abstraction is because people lack up front information to be able to know what the right abstraction is, and how much of it is because choosing good abstractions (in general and specifically ones that are resilient in the face of changing requirements) is a skill that takes work/time/experience/etc. to develop? If it's due to being unable to predict the future, then it makes sense to avoid abstractions. If it's due to not being as good as you could be at creating abstractions, then maybe improving your ability to do so would allow a third option: instead of choosing between duplication and a bad abstraction, maybe you can choose a good abstraction.
- nfw2 6y ago
  1.) I think political and interpersonal issues can play a role here. People are often hesitant to suggest other people's code needs to be rewritten. This is especially true if an abstraction is heavily-used by the organization. If there are many stakeholders using the abstraction, the motivation behind the refactor (ie the perceived defects), would likely need to be communicated widely to justify the effort the refactor requires.
- zbentley 6y ago
  > Why do they feel this way, and is the feeling legitimate? In my experience, it's because the amount of diff (red or green) in a change request is--consciously or subconsciously--correlated with risk. Even though we killed SLoC as a productivity metric years ago, the idea that "change/risk is proportional to diff size" is still pervasive. I'm totally into YAGNI/"code volume is liability" school of thought. But equating change volume with liability is a subtly different and very harmful pattern. Adding a single conditional inside your typical 1200 line mixed-concern business-critical horrorshow function may assume a much greater liability (liability as in bug risk and liability as in risk/difficulty of future changes) than e.g. deleting a bunch of unused branches, or doing a function-extraction refactor pass. Standard "change one thing at a time" good engineering practices still apply of course.
memexy 6y ago
> Re-introduce duplication by inlining the abstracted code back into every caller. Ideally this type of workflow would be supported by the code editor. I've done this manually a few times and it's not fun.
nbardy 6y ago
This has been one of the hardest fought lessons I’ve learned it my programming career, but also one of the most fruitful. I am to make my abstractions too late rather than too early. My rule of thumb tends to me copy things six to seven times before you try to build an abstraction for it.
SkyPuncher 6y ago
I think there's a big cultural challenge with adopting duplication. It goes against most people's career growth objectives. Being able to effectively create clean, re-usable abstractions is a measure of being a "senior" engineer at many places. In other words, to be viewed as senior, you need to be able to effectively write abstraction frequently. It's hard to measure an abstraction in the moment, so a lot of people assume that the senior simply knows better. I find this extends to a lot of programming. Seniors will often use unnecessary tricks or paradigms simply because they can. It can make it extremely difficult for junior developers to grok code. Often this re-enforces seniority. "If only the seniors can work on a section of code, then they are senior". Likewise, there are so many books on crazy architectures and patterns. It's really neat to understand, but I've determined those books are pretty much self-serving. ---- I've found that my work is often far more limited by the domain/business logic than any sort of programming logic. I'll happily write code that looks really basic - because I know ANYBODY can come in and work with that code. If I write code that a junior needs to ask me questions like "what is this pattern?" or "what does this mean?", I've written bad code. ----- With all that being said, every single job interview I've ever had expects me to write code at the level of complexity that my title will be at. They'd much rather see me build some sort of abstract/brittle concept than using some constants and switch statements. The prior looks cool, the latter looks normal.
- leto_ii 6y ago
  > I think there's a big cultural challenge with adopting duplication. It goes against most people's career growth objectives. My experience is the complete opposite :D. What I've noticed is that the people who 'deliver' quickly (without much regard for what might be called code quality) and fulfill business requirements without much questioning are perceived as more valuable. > I've found that my work is often far more limited by the domain/business logic than any sort of programming logic. I broadly agree with this statement. However, just like a good carpenter knows how to properly build a bookcase, a table, a roof etc. a good developer should understand the programming logic and know how to apply it. Business requirements need to be fulfilled, but it's up to us to decide how to do that. More so, I think it's up to us to push back when we feel business requirements don't make sense from a technical point of view, or even from a business point of view.
cjfd 6y ago
If there is one single article about programming that I hate it is this one. It is completely the wrong message. One should instead be very eager to eliminate duplication. To avoid the pitfalls that the article notes one should create abstractions that are the minimal ones required to remove the duplication to avoid over-engineering. Also one should keep improving the abstractions. That way one can turn the abstraction that turned out to be wrong into the right one. It is the attitude of constant improvement that will make one succeed as opposed to the attitude of fear of changing something that this article seems to encourage. When one does things one learns. When one is afraid to try things everything will just calcify until it is no longer possible to add any new features. What one does need to make the refactoring work is automated tests.
- Ensorceled 6y ago
  In 30 years, I can count on the fingers of one hand the number of times I've encountered projects that were in trouble because there was copy/pasted code everywhere and the team was not abstracting out of fear of breaking the existing code. What I have encountered is dozens of projects that had essentially ground to a halt because of numerous deeply, and incorrectly, abstracted systems, modules and libraries. Correcting projects in this state has almost always been refactoring into fewer abstractions; less complex, more cohesive and less coupling.
  - dragonwriter 6y ago
    > In 30 years, I can count on the fingers of one hand the number of times I've encountered projects that were in trouble because there was copy/pasted code everywhere and the team was not abstracting out of fear of breaking the existing code. I think the level of experience where underabstraction is common as opposed to overabstraction is so low that it's uncommon to find a team where that gets through, because even if someone junior is at the level where it's common, they’ll get corrected before it becomes a widespread problem.
  - zbentley 6y ago
    I don't disagree and have seen the same thing. However, I've also noticed in those cases that it's very hard to get people to agree on what the problem actually is. One person's incorrect over-abstraction is another person's incompletely-DRYed-up code.
allenu 6y ago
In a large organization, the other thing you notice with trying to fix duplicated code is, if you take on refactoring it all, you are now responsible to make sure everything still works AND that you do not inhibit any future work. You are now responsible for more than you may have bargained for. Coming up with the right abstraction takes some predicting of future use-cases. It's more than just refactoring work to put it all in one place.
thinkloop 6y ago
A related problem: duplication is not equality. If two things happen to be the same right now, it doesn't mean they are intrinsically the same thing. If you have multiple products selling for $59.99 they shouldn't share a function to generate the "duplicate" price. Abstractions needs to be driven by conceptual equivalence, not value equivalence, where duplication is a good hint for a potential candidate of abstraction, but not the complete answer alone.
tomphoolery 6y ago
This again?? ;) I love this post. A lot of wasted hours were spent in the past trying to use abstractions that no longer made sense, but Sandi encouraged me to go back and rethink a lot of that and now my code is way easier to read. Thanks Sandi!
kureikain 6y ago
I think one of the cool thing about pattern matching or language(In my case, it's Elixir) that support function operator is we can have same method with different argument sigunatures. So we don't have to duplicate or inherit whatever and still share some common method.
haolez 6y ago
That's mostly how I matured as a developer: I find myself abstracting less and writing less code today than I did 10 years ago, but I'm more productive today, my code is cheaper to maintain and has fewer bugs. Sometimes, I will literally copy paste a small amount of logic just to avoid making a future reader of this code to keep hunting around where the business logic is actually implemented. "It's right here, my dear future reader!". Or maybe I was just a really bad programmer 10 years go :)
kristo 6y ago
There should be a code tool to re-inline code from an abstraction
worik 6y ago
Really this is stating the obvious. The social problem at step 6, 7, and 8 is a social and economic one. Having the time, resources, and skill to do a job properly is very important. But there are social and economic pressures to "just get it done". This is a specific formulation of a general problem.
avodonosov 6y ago
> they alter the code to take a parameter, and then add logic to conditionally do the right thing based on the value of that parameter But that's a textbook example of bad code, competent coders don't do this. Update: for example see Thinking Forth chapter "Factoring Techniques", around the tip "Don’t pass control flags downward.". Page 174 in the onscreen PDF downloadable from sourceforge. And there is no need for duplication. The bigger function can be split into several parts so that instead of one call with flag everyone calls needed set of smaller functions.
- dragonwriter 6y ago
  Competent coders do suboptimal things all the time, especially when there is delivery pressure; competent doesn't mean infallible or perfect. There's also not a clear boundary between what is a single appropriate abstraction and two (or N) distinct but superficially related concepts.
- zbentley 6y ago
  > that's a textbook example of bad code, competent coders don't do this. That's reductive and dismissive. There's a ton of subtlety in even defining the terms for that "best practice". What counts as a control flag versus a necessary choice that must be made by callers? Are you still passing control flags if you combine them into a settings object? What if you use a builder pattern to configure flags before invoking the business logic--is that better/worse/the same? What if you capture settings inside a closure and pass that around as a callback? How far "downward" is too far? How far is not far enough (e.g. all callers are inlining every decision point)? The answer to all of those is, of course, "it depends on a lot of things". And that's before you even get into the reality (which a sibling comment pointed out) that even if we grant that this is inherently bad code, that doesn't imply anything about the competence of the coder--some folks aren't put in positions where they can do a good job. Unrelated aside: Thinking Forth is an excellent book! Easy to jump into/out of in a "bite size" way, applicable to all sorts of programming, not just Forth programming.
vxNsr 6y ago
I want to thank everyone here, I’ve been stuck for about a week now on an issue that is entirely germane to this topic and the whole conversation here really helped me flesh out what was wrong and allowed me to understand a path forward. I’m honestly holding myself back from popping onto my computer right now to start working on it.
gm 6y ago
This advice just _feels_ very wrong. After thinking about it and seeing the other comments, some remarks: 1) It's fine to go back and duplicate code after you correct the abstraction. But it should be the _first_ phase in doing a larger pass to refactor code to fit the current business requirements. If you forgo the _second_ step, which should be to search for suitable abstractions again, you are absolutely guaranteed to be left with shit code that breaks in this situation, but not that other one, and no one knows why. I would absolutely only duplicate code as the prequel to deduplicating it again with updated abstractions. 2) If you do any of this without thorough unit tests you're insane. Keep the wrongly-abstracted code unless you have time to thoroughly fix the mess you will have made when you duplicate code again and introduce bugs (you're human, after all). 2a) If you are going to do this and there are no unit tests, create those unit tests before you touch the code initially (before the duplication). 3) Some of the comments saying you should wait until you implement something two or three times before creating an abstraction seem like comp sci 101 rules of thumb. It's way too simplistic a rule, way too general. Prematurely abstracted (haha!). The type of project and the type of company/industry will tell you what the right tradeoff is. That is all.
- haolez 6y ago
  You are assuming that the code is a moving target. Not every software project behaves that way. Sometimes, the software gets done as is.
  - gm 6y ago
    In that case, then the original problem (incorrect abstraction) does not exist, or at least does not get worse over time, and thus does not need fixing.
- roryokane 6y ago
  The article already agrees with you on point 1: > Once you completely remove the old abstraction you can start anew, re-isolating duplication and re-extracting abstractions.
nfw2 6y ago
When considering whether some abstraction is "right" or "wrong", another important thing to consider is how cleanly the abstraction fits into a mental model of how the program works. Good abstractions provide value outside of removing duplication. They help us reason about a program by providing compression of logical concepts. Consider some helper function: "convertSnakeToCamelCase." This abstraction would take a string, do some operations on it, and return another string. It is easy to understand what the input and output is without having to think about these operations. This abstraction provides a benefit for anyone having to think about the program because it reduces the amount of concepts the reader has to parse from N (where N is the number of operations) to 1. This is helpful because people have limited mental bandwidth and can only reason with a finite number of concepts at any given time. Consider a different helper function: “processDataPayload.” This function takes data in some arbitrary complex shape and returns data in some arbitrary complex shape. The abstraction effectively communicates nothing to the reader, and it is actively unhelpful because it forces that person to follow a reference, remember all the details of what that function does, and substitute those details into the original function. Trying to find the conceptual boundaries that make the program easiest to reason about IMO is more of an art than a science and difficult to govern with hard and fast rules.
- jasonhansel 6y ago
  Agreed. I also think it's important to create abstractions that provide guarantees and/or maintain invariants. That way, your abstractions actually help you be more confident that your code is correct. The point of abstraction isn't per se to reduce duplication--it's to make your code more straightforward and to make errors more obvious.
recroad 6y ago
Programmer B in Step 6 should have used SOLID and refactored to extend the module (or something similar). This is strawman argument which has little to do with the "wrong" abstraction and everything to do with poor design choices.
kuharich 6y ago
Prior comments: https://news.ycombinator.com/item?id=17578714 https://news.ycombinator.com/item?id=17578714
ragnese 6y ago
I can't help but wonder if we're sometimes using the wrong words for things. In this discussion we keep talking about "code duplication" and "abstraction" hand-in-hand, but I think they're almost orthogonal concepts, at least as I think of them. Seeing the same code almost copy-and-pasted in a few places might call for some code-deduplication. But that's not necessarily a new "abstraction" in my eyes. It may be, but it also may not be. I'm struggling to think of a specific example because I fully intended to go to bed before arriving here... But as a really stupid example, let's say you have `val a = x + x + x` and `val b = y + y + y` and `val c = z + z + z` in your code. If you write a new function like `fun addThreeTimes(i) = i + i + i`, I don't see that as a new abstraction at all. If, however, you invent multiplication, now you're at a new abstraction! `val a = x * 3; val b = y * 3`, etc. "Abstraction" to me is about thinking at a different semantic level, not about avoiding copy and paste. Does this resonate with anyone else? Am I missing the point?
- lambdatronics 6y ago
  I would say it becomes an abstraction when it needs parameters in addition to the nominal arguments. So in the case of "addThreeTimes(i)", sure, that's just abbreviation, but "addNtimes(i,N)" is a bona-fide abstraction. Edit: Reminds me of the distinction between inheritance and subtyping: https://www.cs.utexas.edu/users/wcook/papers/InheritanceSubtyping90/CookPOPL90.pdf https://www.cs.utexas.edu/users/wcook/papers/InheritanceSubt...
  - ragnese 6y ago
    I agree with your point and your edit. The inheritance thing is very similar to what I'm talking about!
- epage 6y ago
  In my experience, code dedupping without shared requiremeots hurts you over time. Nothing is keeping the code focused so as requirements change, you either fork it or add "one more flag". Focusing on abstractions is what helps me in dedupping with a focus on requirents.
- MaulingMonkey 6y ago
ricksharp 6y ago
The mistake is creating an abstraction because of seeing duplication. DRY is not a good guiding principle. It is an anti-principle. Abstractions should only be created when they have a clear purpose and create a simpler architecture by encapsulating a single concern. The reality is that all code is duplication. The reason we write code is because it is the most concise language to specify the intended goal in the current context. What is unique is not the code that we are writing. The unique part is the code in the current context and each level of abstraction separates the context from the implementation - so that abstraction must be beneficial in organizing the overall solution into individual logical components of singular concern.
runald 6y ago
For something that argues against bad abstractions, the article sure is lacking in concrete examples and makes a point in abstract. A lot of people will likely misinterpret or get the idea that abstraction is only done for duplicated code (DRY as some people would call it). I think the wrong/bad "abstractions" here mostly refers to abstraction that was made over common code that is very specific in a context and is very susceptible to domain changes. But there are a lot of other kinds of abstraction aside from DRY. There are abstractions made to reduce clutter and hide implementation detail and will likely be used only once. There are also abstraction that are more general and aren't coupled to the domain. These abstractions are more reusable and composable, and are immune to domain changes such as the step 6 in the article. Some people would find these kinds of abstractions harder to digest, but I personally consider these kinds of abstractions as extensions to the standard library, or even additions to the vocabulary of the programming language. Note that I don't claim that general abstractions are necessarily better, since the generality can be made to the extreme and we'd have monads for breakfast. All in all, I agree with the article, except that it is only referring to one kind of abstraction, although I hesitate to call it as such.
ninetax 6y ago
What are people's recommendations on books on how and when to create the right abstractions? Last year I read Zach Tellman's _Elements of Clojure_ and really loved the parts that touched on the subject of abstraction.
neetrain 6y ago
I think the term "wrong" causes all the misunderstandings. It sounds like the abstraction was wrong in the first place. Can it be called "rotten" abstraction?

654 points by mkchoi212 6y ago

I’m not sure why this is #1... but since it is, both of these - duplication and wrong abstractions - are otherwise known as technical debt.

dasil003 6y ago
Not necessarily. Technical debt is when you do something quick and dirty to get a feature out in the short-term knowing that it won't be maintainable, scalable, etc, but you do it anyway with the expectation that you'll fix it later. Some duplication and wrong abstractions are caused by this, but definitely not all.
- hrhrhrd 6y ago
  No, technical debt is a very general category that includes deliberate hacks, structural flaws, and small mistake bugs. It's anything that over time will damage the code base, duplications and wrong abstractions being very much included in that
  - dasil003 6y ago
    You're welcome to your own definitions, but personally I keep bitrot, deferred maintenance, and "structural flaws" (which can be subjective and dependent on use cases and scale) out of the bucket of technical debt since it robs the metaphor of a defining aspect: intentionality. Debt is not something that happens passively as the world changes around you, it's something which you sign up for.
    - quinnirill 6y ago
      If you unintentionally destroy property and have to pay for it, you’re in debt. We even have a concept of life debt. Some debt is intentional, some incidental. Most technical debt I’ve seen was not intentional, just a well meaning design that was created to serve a purpose that eventually outgrew it, and that’s when the interest started to pile up. And happening passively is exactly what it does, interest rates change, your ability to make downpayments change. All part of the very well functioning metaphor in this context.

twirlock 6y ago

cc my know-it-all coworker

chiefalchemist 6y ago

Why not simply duplicate the abstraction, refactor as needed, and adjust the necessary caller(s)? Having to know, find and maintain the individual duplications feels dirty and its own way wrong. Choose your wrongs wisely?

Pxtl 6y ago

Every Line Of Business codebase I've worked on has been the worst "there I fixed it" copypasta spaghetti, and has never made it to the point where "maybe we shouldn't add a parameter to this existing, cleanly abstracted method to handle this new similar-but-distinct use-case" was anywhere near my radar for abstraction. I would love to have developers where my problem was "maybe you piggybacked on existing code too much, in this case you should've split out your own function".

mlthoughts2018 6y ago
It’s been the exact opposite for me. The spaghetti code has always come from poorly conceived abstractions and the massive problem of inverting an API to reimplement functionality through the API that should be extensible within the API (but fails to be because of poor choices in abstraction or abstracting prematurely). Later on that spaghetti code gets labeled as lacking abstraction, similar to what you are saying, despite the actual problem being too much abstraction and poorly designed abstraction that became load bearing in a way where everyone decides that living with API inversion as a reality is the lesser evil and figures they’ll probably quit the company and move on to greener pastures before it becomes their headache to deal with. https://en.m.wikipedia.org/wiki/Abstraction_inversion https://en.m.wikipedia.org/wiki/Abstraction_inversion
- hesdeadjim 6y ago
  Absolutely this. I’d rather look at 200 lines of linear, inline documented code then a spaghetti mess of “helper” functions that do nothing better than obfuscate everything going on. I’ve had a strict rule with my team of “1, 2, N”. I don’t want to see an abstraction until we’ve solved a problem similarly at least two times, and even then an abstraction may still be a poor idea. Abstraction is an especially poor idea early in a project because often you only half know what you’re making (I’m in games). Requirements change, or a special case needs to be added, and all of a sudden you are trying to jam new behavior into “generic” helpers without breaking the house of cards built around them.
  - hackinthebochs 6y ago

goto11 6y ago

I'm skeptical because it is really easy to un-share code by copying it into multiple places but it is very hard to unify duplicated code. So I prefer to err on the side of sharing. But yes, you should be ready to change sharing into duplication if you realize the code is just "accidentally similar" and need to evolve in separate directions. In practice I have seen a lot more pain due to duplicate code compared to the issue of over-abstracting code, because the latter is much easier to fix.

joeframbach 6y ago
On the other hand, it's really difficult to know who is using that shared code. If you make an innocuous change in a shared method, it could affect someone else you don't know.
- TheCoelacanth 6y ago
  It's very easy with proper tooling.
- BoiledCabbage 6y ago
  Outside of publishing a public API almost any modern language and enviroment should make this easy.
- bcrosby95 6y ago
  It's a million times easier than figuring out if those minor differences in duplicate code are accidental or on purpose. As bad as a flag-laden method might be, you know the intent of all callers.
- mcintyre1994 6y ago
  I find it much easier to find the call sites for a function than to find code that’s duplicating or a variant of the code I just fixed a bug in so we can figure out if the same bug is latent in the duplicates too.
- dtech 6y ago
  Not in any modern language or IDE. Not to mention that would indicate a hole in the test suite

gorgoiler 6y ago

Brilliant insight. Always remember: (1) make it work, (2) make it right, (3) make it fast. 80% of projects get scrapped in between (1) and (2) because you end up realizing you wanted something completely different anyway.

nicoburns 6y ago
On my projects code doesn't make it into the main branch until it gets to at least (2).
willcipriano 6y ago
> (1) make it work, (2) make it right, (3) make it fast. I've always disagreed with this. In my view you should make it a habit to write optimized code. This isn't agonizing over minor implementation details but keeping in mind the time complexity of whatever you are writing and working towards a optimal solution from the start. You should know what abstractions in your language are expensive and avoid them. You should know roughly the purpose of a database table you create and add the indexes that make sense even if you don't intend to use them right away. You should know that thousands of method lookups in a tight loop will be slow. You should have a feel for "this is a problem someone else probably solved, is there a optimal implementation I can find somewhere?". You should know when you use a value often and cache it to start with. Over time the gap between writing unoptimized and mostly optimized code gets smaller and smaller just like practice improves any skill.
- criddell 6y ago
  > You should know that thousands of method lookups in a tight loop will be slow. That's not always the case. Modern compilers do a lot of things like inlining and unrolling. These days I mostly try to write code that is easy to understand.
  - willcipriano 6y ago
    > Modern compilers do a lot of things like inlining and unrolling Smart ones do, I've been writing Java lately and that behavior tends to be unpredictable and rare[0]. I'd use a inline keyword if I had one, or preprocessor directive of some kind if I had that but I don't. I agree it's harder to read but I feel like changing a JVM flag to get a behavior that I want is more inscrutable than having a long method with a comment noting that this in inlined for performance reasons. With modern machines and the price of memory I tend to lean hard to the memory side of the time-memory tradeoff. [0]"First, it uses counters to keep track of how many times we invoke the method. When the method is called more than a specific number of times, it becomes “hot”. This threshold is set to 10,000 by default, but we can configure it via the JVM flag during Java startup. We definitely don't want to inline everything since it would be time-consuming and would produce a huge bytecode." https://www.baeldung.com/jvm-method-inlining https://www.baeldung.com/jvm-method-inlining

ridaj 6y ago

Previously discussed here: https://news.ycombinator.com/item?id=17578714 https://news.ycombinator.com/item?id=17578714

arendtio 6y ago
I find that first comment particularly insightful. However, I am not sure about the order of state and coupling. To me it seems to depend on the language, as for functional languages, avoiding state is king and in object oriented environments, coupling could be a more important factor.

kolinko 6y ago

I wish this article was available two years ago when I tried to explain this to a bunch of juniors working for me...

nnutter 6y ago
“ Posted on January 20, 2016 by Sandi Metz.”
- kolinko 6y ago
  Damn, I wish I saw it back then :)

brandonmenc 6y ago

Junior programmers duplicate everything. Intermediate programmers try to abstract away absolutely every line that occurs more than once. Expert programmers know when to abstract and when to just let it be and duplicate.

leafboi 6y ago
The master never duplicates and all his abstractions are intuitive, readable and flexible.

kevsim 6y ago

Relevant post from earlier today https://news.ycombinator.com/item?id=23735991 https://news.ycombinator.com/item?id=23735991

klyrs 6y ago

I use DRY in two ways. The first is that I'm happy to make 2 or 3 copies of a snippet before promoting that to a new function. The second is when I find a bug in a duplicated snippet. I'll mend the snippet and its duplicates, once or twice before promoting it to a function. In the rarer (in my line of work) instance that a common snippet gets used with several intrusive variations, I usually document the pattern. It's tempting to use templates, lambda functions, closures, coroutines, etc but far simpler to duplicate the code. But again, if a bug (or refactor) crops up and I need to fix it in many places, then I'll spend some time thinking about abstraction and weigh the options with the benefit of hindsight.

jpswade 6y ago

You can’t plan for what you don’t know. This is why I like the "Rule of three"[1]. Only once you've done it three times will you truly begin to understand what the abstraction might need to look like. 1. https://wade.be/2019/12/10/rule-of-three.html https://wade.be/2019/12/10/rule-of-three.html

geophile 6y ago
Exactly. With experience, you learn not to abstract too soon.
- hesdeadjim 6y ago
  Seems so counterintuitive, but it’s one of the most important lessons I’ve learned in 15 years of development experience.
deleted 6y ago
[deleted]
ed312 6y ago
Any advice on teaching this to junior engineers? Seems like folks with 3-5 years of experience keep trying to not only over-abstract but also keep re-inventing the wheel with abstractions (vs looking for existing libraries).
- deleted 6y ago
  [deleted]
- leafboi 6y ago
  It's largely because they're dealing with an area with no theoretical tools. Any time you hit an area that are full of people "Designing" solutions/abstractions rather then "Calculating" an optimal solution/abstraction you know you've hit an area where there's very little theoretical knowledge and most people are just sort of wandering chaotically in circles trying to find an "optimal" solution/abstraction without even a formal definition of what "optimal" is.... I mean what is the exact definition of the "perfect abstraction"? What is bad about duplication what is a bad over abstraction and what is this "cheaper" cost that the title is talking about? It's all a bunch of words with fuzzy meanings injected with peoples biased opinions. That being said theories on abstractions do exist. If you learn it you'll be at the top of your game; but it's really really hard to master. If you do master it, you'll be part of a select group of unrecognized elites in a world of programmers that largely turn to "design" while eschewing theory. Here are two resources to get you started: The Algebra of Programming: https://themattchan.com/docs/algprog.pdf https://themattchan.com/docs/algprog.pdf Program Design by Calculation: http://www4.di.uminho.pt/~jno/ps/pdbc.pdf http://www4.di.uminho.pt/~jno/ps/pdbc.pdf You will note that both of these resources talk about functional programming at its core which should indicate to you that the path to the most optimal abstraction lies with the functional style.

seanalltogether 6y ago

This quote from John Carmack speaks very succinctly to the problems that many abstractions in a code base can cause, and it's a constant reminder for me when building out business logic. > "A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in." https://www.gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php https://www.gamasutra.com/view/news/169296/Indepth_Functiona...

hackinthebochs 6y ago
But abstractions reduce possible state and allows you to specify that state in obvious ways, e.g. on function parameters. Do not underestimate the power of functional boundaries.
- ben509 6y ago
  They also tend to impose a degree of discipline. I've often found myself wanting to shove a parameter in somewhere and realized I didn't need the damned thing.
hesdeadjim 6y ago
This is one reason I love working in the Unity ECS framework. Your data is public and state can’t hide. Your systems are still free to contain a plethora of bugs, but they are easier to track down due to the functional nature of a system. In the regular Unity OOP land, developers inevitably sprinkle state everywhere. Coroutines are by far one of the worst offenders. Good luck seeing the current executing state of your game when it’s hidden in local variables inside a persistent function body...
jonnycat 6y ago
Reading that article and the context of the quote, it appears that Carmack is using that statement to extol the benefits of functional programming styles, not commenting on abstraction.
- seanalltogether 6y ago

pps43 6y ago

Related to http://yosefk.com/blog/redundancy-vs-dependencies-which-is-worse.html http://yosefk.com/blog/redundancy-vs-dependencies-which-is-w...

deleted 6y ago

[deleted]

crazygringo 6y ago

Another tip is: if you're duplicating, and they're not lines of code that are visually obviously next to each other, then leave a comment next to both instances mentioning the existence of the other. There's nothing inherently wrong with duplication, except that if you change or fix a bug in one, you need to not forget about the other. Creating a single function solves this... but at the potential cost of creating the wrong abstraction. When you're at only 1 or 2 extra instances of the code, just maintaining a "pointer" to the other case(s) with a comment serves the same purpose. (Of course, this requires discipline to always include the comments, and to always follow them when making a change.)

stormdennis 6y ago
Would the risk forgetting to update the comments not be a reason for creating a wrapper method that handled calls to both and contained the relevant advice?

deleted 6y ago

[deleted]

djhaskin987 6y ago

Mods this article is old, should be labeled 2016.

gumby 6y ago

Early de duplication is the equivalent of early optimization: a bad idea that boxes you in. Duplicate code is a sign that there could be a generalization missing.

tarkin2 6y ago

"With C you can shoot your own foot. With C++ you can blow your own leg off". I feel the same is true here. The abstraction may be right at the time of writing, yet further on it often becomes not only wrong, but a massive hindrance. With time and effort, hacky code and be worked into shape. An eventual wrong abstraction normally means a rewrite.

random3 6y ago

This is so true, but so shallow too. I think the big mistake is to treat the code as "the main thing" when in reality it's just a model (a golem) mimicking some "other thing" We're missing an entire set of code characterizations. Yes we have a "pattern language" but there's not much to characterize it structurally wrt "code distance" from one part of the code to the other (e.g. in call stack depth as well as in breadth). And again all of this needs to happen wrt the "abstraction" not the code itself. Having 10 methods 90% duplicated in a single file with 10% pecent difference is many times better than trying to abstract it. Having the same "unit conversion" function duplicated in 3 parts of the code can be disastrous. These two examples are very easy to see and understand, but in reality you're always in a continuous state in between. And "code smells" like passing too many parameters or doing "blast radius" for certain code changes are only watching for side-effects of a missing "code theory". An interesting book on the topic is "Your code as a crime scene". The bottom line is we're trying to fix these problems over and over again without having a good understanding of what the real problem is and this leads to too many rules too easy to misinterpret unless you are already a "senior artist"

ijidak 6y ago
> Having the same "unit conversion" function duplicated in 3 parts of the code can be disastrous. This. I feel like it's really about cognitive load to remember and recognize the differences. Duplication in 3 distant files, places a heavy load on the developer to: 1. Discover the duplication 2. To grasp the reason for the differences in the 3 different locations. 3. Remember these things Whereas when the duplication is in the SAME file, #1, #2, and #3 can become very manageable cognitively. Now the question changes to.. Is the cognitive load of dealing with the different special cases in a single de-duplicated method GREATER than simply leaving them in separate methods? Often the answer is duplication WITHIN a file is less of a cognitive load. Whereas duplication ACROSS files is a heavy cognitive load. Minimizing cognitive load minimizes mistakes. And minimizes developer fatigue. Thus boosting productivity. At least, that's my development philosophy, even though I've never seen it in a design pattern or a book. It just seems to make sense.

bcrosby95 6y ago

I find it interesting that comments on these articles mainly discuss 1 aspect about it. But rarely this part: > Don't get trapped by the sunk cost fallacy. In my experience, yes, programmers are hesitant to throw out an abstraction. Why not work to change this, rather than telling people not to abstract?

ben509 6y ago
I don't think it's a sunk cost fallacy. I think the hesitation is more for social reasons, often not wanting to do a big pull request that's going to be scrutinized.
- Tainnor 6y ago
  "Big pull requests" that are unannounced are always problematic because who wants to be the person saying "all of this work you've done is wrong"? In such situations, it's good to get buy-in from other people before attempting to do such a thing. Make a proposal for a big change and discuss it. There's still a chance that, in the implementation it doesn't work as nicely as believed initially, but at least now it's less likely that the idea will be rejected wholesale during code review.

naringas 6y ago

sometimes it's better to copy and paste some code only to make each copy diverge more and more over time (somewhat like a starting template) as opposed to introduction an abstraction to generalize some slightly different behaviors only to use said abstraction twice. this makes even more sense when the code will live on in different programs there's a point when incurring the cognitive overhead costs of the abstraction become worthwhile, probably after the 3rd time. but my point is that it's also important to consider that the abstraction introduces some coupling between the parts of the code.

rightbyte 6y ago
I find it easier to read long functions of code than jumping around in helper functions or abstractions. Especially if I am not familiar with the code base and don't know common functions by heart.

adamkl 6y ago

Sandi mentions this during a talk she gave on refactoring a few years ago. [0] It’s a great little video for showing junior developers how a messy bit of code can be cleaned up with a few well chosen OOP patterns (and a set of unit tests to cover your ass). [0] https://youtu.be/8bZh5LMaSmE https://youtu.be/8bZh5LMaSmE

voodoologic 6y ago
I'm very partial to this talk about proper abstraction (and not just for junior developers): https://www.youtube.com/watch?v=OMPfEXIlTVE https://www.youtube.com/watch?v=OMPfEXIlTVE

ulisesrmzroche 6y ago

“Premature optimization is the root of all evil”

Xlurker 6y ago

I'd rather ctrl-f and change code in multiple places than deal with abstraction hell.

pierrebai 6y ago

Counter: Refactoring is far, far, far cheaper than duplication or wrong abstraction. Duplication means you lose the wisdom that was gained when the abstraction was written. It means that any bug or weird cases will now only be fixed in one place and stay incorrect for all the places you duplicated the code. About the rule of three: I personally extract functions for single-use cases all the time. The goal is to make the caller be as close to pseudo-code as possible. Then if a slightly different case comes up, I will write the slightly different case as another function right next to the original one. Otherwise, the fact that you have multiple similar cases will be lost.

fiddlerwoaroof 6y ago
Yeah, the rule of three is misleading: having a name for three lines of code that do “one thing” is almost always a win and nothing prevents a future developer from either inlining that function, if it was a bad idea, or duplicating and modifying the function.
twirlock 6y ago
Yes, everyone on the planet understands why duplication is not ideal. Literally nobody entered the discussion not knowing that. You clearly don't grasp what abstraction is fyi.
jonahx 6y ago
Counter-counter: Refactoring is by far the most expensive and error prone activity in programming. It can also be one of the most valuable. But unless it's trivial, it's the most mentally arduous and time-consuming work you do as a programmer.
- titzer 6y ago
  I disagree. I think debugging is the most expensive activity a programmer ever does. Refactoring is a luxury that you have when you don't have bugs or time pressure to ship/fix something. Debugging potentially requires you to load the entire context of the (incorrect) program into your head, including irrelevant parts, as you grope around to figure out a.) what actually went wrong, b.) why it went wrong, and c.) how to modify the existing system in a way that doesn't make it worse. Debugging is reverse-engineering under the gun. It has huge cognitive load. Especially debugging a production system with a difficult to reproduce bug in a deep dark part of the code. It's a nightmare scenario. Refactoring, on the other hand, often happens with incomplete knowledge, and can be quite local. I've seen zillions of refactorings that are done with incomplete knowledge that are local improvements (and many that were not global improvements).

Tainnor 6y ago

I feel some people here are misunderstanding the blog post. Sandi Metz IMHO doesn't claim that the problem occurs at step 2 or 3. She doesn't claim that it's wrong to introduce abstraction when there is duplication. What she is saying instead is that the problem occurs from step 6 onwards: when you find yourself wanting to reuse an abstraction that, regardless of whether it made sense in the first place or not, has outlived its usefulness. I think this is in agreement with other points that she often makes, about being bold, but methodical about refactorings. The whole discussion about "you should never abstract away code before you see the third duplication" has little to do with the article, and I'm also really not sure it's good advice.

kwhitefoot 6y ago
> Sandi Metz IMHO doesn't claim that the problem occurs at step 2 or 3. But the headline does. I had to read quite a long way down the page to discover that all she is advocating is what i have always done: deduplicate when practical, undo the duplication when new requirements make it incorrect and push the unique parts into the callers.
- DougBTX 6y ago
  > But the headline does. That’s not really fair, it repeatedly says “wrong abstraction”, in the title and in the article. At steps 2 and 3 it is still the right abstraction, duplication only becomes better when it is the wrong abstraction.
tarsinge 6y ago
That’s not what I get from the article. The problem does indeed occurs at step 2 and 3: leave duplication alone and don’t introduce abstraction if you are not sure about future requirements.
- Chris_Newton 6y ago
  Taken to its logical conclusion, doesn’t that argument mean we would almost never introduce any abstractions at all? That doesn’t seem very practical compared to the alternative of introducing abstractions if they are useful at the time but remaining willing to change or remove them again later if the situation changes.

hackinthebochs 6y ago

I couldn't disagree more. There is no such thing as abstracting too early (this does not go for structural abstractions like factories, singletons, etc). The best code is code you don't have to read because of strong, well-named functional boundaries.

amelius 6y ago

A manager once asked me: please reuse as much code as you possibly can. This reminded me of that.

deleted 6y ago

[deleted]

hota_mazi 6y ago

> prefer duplication over the wrong abstraction Such a strange advice. If you're able to recognize the wrong abstraction right away, surely you would not use it, right?

mrkeen 6y ago
The reason is that you won't know it's the wrong abstraction until it's time to modify it or add new features.
allenu 6y ago
I think the intent was to communicate that abstractions aren't always right. Some people might think that because there's duplicate code and that the abstracted code maps to the duplicated code 1 to 1 and leads to fewer lines in total, it's a good abstraction, not realizing that there are costs to doing this that may not be aware of.

jack_h 6y ago

I would say that if developers are hacking on an abstraction that is ill-suited to the task until the code base is a nightmare, they will take this advice and duplicate code until it's a nightmare. The fact of the matter is every line of code that is written has an associated cost. Developers all too often pay that cost by incurring technical debt.

bob1029 6y ago

This whole thing exists on a normalized/de-normalized spectrum. The problem is that both ends have pros/cons. On the normalized side, you have the benefit of single-point-of-touch and enforcement of a standard implementation. This can make code maintenance easier if used in the correct places. It can make code maintenance a living nightmare if you try to normalize too many contexts into one method. If you find yourself 10 layers deep in a conditional statement trying to determine specific context, you may be better off with some degree of de-normalization (duplication). On the de-normalized side, you have the benefit of specific, scoped implementations. Models and logic pertain more specifically to a particular domain or function. This can make reasoning with complex logic much easier as you are able to deal with specific business processes in isolation. You will likely see fewer conditionals in de-normalized codesites. Obvious downsides are that if you need to fix a bug with some piece of logic and 100 different features implement that separately, you can wind up with a nasty code maintenance session. I find that a careful combination of both of these ideas results in the most ideal application. Stateless common code abstractions which cross-cut stateful, feature-specific code abstractions seems to be the Goldilocks for our most complicated software.

why-el 6y ago

Rob Pike discusses similar points in this section of his talk on Go Proverbs https://www.youtube.com/watch?v=PAAkCSZUG1c&t=9m28s https://www.youtube.com/watch?v=PAAkCSZUG1c&t=9m28s.

dfischer 6y ago

Reminds me of this discussion: https://news.ycombinator.com/item?id=12120752 https://news.ycombinator.com/item?id=12120752 (John Carmack on inlined code).

jbmsf 6y ago

One of the reasons duplication is used badly is that it is one of the easiest abstractions to recognize. One of the ways I've seen DRY go horribly wrong involves reusable code units evolving into shared dependencies that often interdepend in complex ways. Unfortunately, the problems of such a system are observed much later than the original code duplication and fewer people have the experience to see it coming.

leto_ii 6y ago

As I gain more and more experience (I would now call myself more or less a mid-level developer), I find that the distinction that matters is not abstraction vs duplication, but the one between developer mindsets. I have many times met/worked with people who think the main task of the developer is to 'get shit done'. Regardless of their level of experience, these developers will churn out code and close tickets quite fast, with very little regard for abstraction, design, code reuse etc. Conversely, the approach that I feel more and more is the correct one is to treat development as primarily a mental task. Something that you first think about for a while and try to design a little. The actual typing will in this case be a secondary activity. Of course, this doesn't mean you shouldn't iterate on your design if during execution problems come up. Just that the 'thinking' part should come before the 'doing'. My feeling is that with this second approach the abstraction/duplication trade-off will not matter so much anymore. With enough experience you will figure out what you can duplicate and what you can design. And when you design you will develop an understanding of how far you should go. Approaching development as a task of simple execution I think inevitably leads to illegible spaghetti down the line.

Tainnor 6y ago
I agree that many issues with bad code could really be avoided by first thinking about the solution a bit, of which the code is just an expression. I'm not advocating weeks of architecture astronauting without code feedback - because practical considerations (e.g. the compiler can't deal with this kind of code due to some limitations) matter - but some people seem overeager to just start writing some code "and see what happens".

scrozart 6y ago

DRY gets abused regularly in my experience. It doesn't stop at method/class abstractions either; I've seen entire microservices & plugins developed to ensure each app doesn't have that one chunk of auth code, for instance, even though they each may have subtly different requirements (those extra params again). The logical end to this sort of thing is infinitely flexible/generic multipurpose code, when the solution is really, probably increased specificity. DRY is probably the lowest-hanging fruit for practices/patterns, and I think this leads to a disproportionate focus on it.

hesdeadjim 6y ago
It’s also easy compared to solving new problems, so it can be an emotionally safe way of feeling productive. Failure is difficult to measure until the abstraction falls flat on its face months later, at which point it can be chalked up to the demons of “changing requirements”.
- zbentley 6y ago
  That is a very, very important point; well put. The "of course it sucks: changing requirements!" boogeyman means one of two things: "the code was written to do the wrong thing because requirements changed/weren't communicated" or "the code was hard to change when it needed to do a new thing". Figuring out which of those two is in play is very important.

preommr 6y ago

I strongly dislike this article because the title is much broader than most of the substance of the article. Advising not to overextend an abstraction is inarguable. The actual title "Duplication is far cheaper than the wrong abstraction", and the thing that people will really discuss, is a loaded statement that's going to need a lot of caveats.

deleted 6y ago
[deleted]

zarathustreal 6y ago

I’ve seen this “hot take” a few times before and even see developers that I would have considered very good agree with it. Consider that all code is computation, this is the point of a computer: to compute. Consider that abstraction doesn’t seem valuable -to you- for a multitude of reasons. Perhaps you’re using a flawed paradigm that emphasizes objects over computation. This would obviously mean abstraction -increases- the difficulty of reasoning about your code. Perhaps you don’t have a mental map of appropriate abstractions due to a lack of education or knowledge gap, this could lead you down the path of creating abstractions which reduce duplicate characters or lines of text but are not logically sound (“leaky abstractions.”) All of these things come together in a modern “enterprise” software environment in just the right way such that abstraction starts to seem like a bad idea. Do not fall into this line of thinking. Study functional programming. Study algebraic structures. Eventually the computer science will start to make sense.

layer8 6y ago

The main takeaway from the article is that abstractions which have become inadequate should be corrected (removed and/or replaced by adequate ones) as soon as possible. A corollary is that abstractions should be designed such that they can be replaced or removed without too much difficulty. A common problem in legacy code bases is not just that they contain many inadequate abstractions, but that the abstractions are entangled with each other such that changing one requires changing a dozen others. You start pulling at one end and eventually realize that it’s all one large Gordian knot. One thing that I learned the hard way over the years is to design abstractions as loosely coupled and as independent from each other as possible. Then it becomes more practical to replace them when needed.

adrianmonk 6y ago

Two questions (genuine, not rhetorical): (1) How much of this is because it's actually hard to back out of the wrong abstraction and pivot to the correct one, and how much of it is other causes? The article hints at this with, "Programmer B feels honor-bound to retain the existing abstraction." Why do they feel this way, and is the feeling legitimate? Do they lack the deep understanding to make the change, or are they not rewarded for it, or are they unwilling to take ownership, or is it some other reason? I could see it going either way, but the point is to understand whether you're really stuck with that abstraction or not. (2) How much of the wrong abstraction is because people lack up front information to be able to know what the right abstraction is, and how much of it is because choosing good abstractions (in general and specifically ones that are resilient in the face of changing requirements) is a skill that takes work/time/experience/etc. to develop? If it's due to being unable to predict the future, then it makes sense to avoid abstractions. If it's due to not being as good as you could be at creating abstractions, then maybe improving your ability to do so would allow a third option: instead of choosing between duplication and a bad abstraction, maybe you can choose a good abstraction.

nfw2 6y ago
1.) I think political and interpersonal issues can play a role here. People are often hesitant to suggest other people's code needs to be rewritten. This is especially true if an abstraction is heavily-used by the organization. If there are many stakeholders using the abstraction, the motivation behind the refactor (ie the perceived defects), would likely need to be communicated widely to justify the effort the refactor requires.
zbentley 6y ago
> Why do they feel this way, and is the feeling legitimate? In my experience, it's because the amount of diff (red or green) in a change request is--consciously or subconsciously--correlated with risk. Even though we killed SLoC as a productivity metric years ago, the idea that "change/risk is proportional to diff size" is still pervasive. I'm totally into YAGNI/"code volume is liability" school of thought. But equating change volume with liability is a subtly different and very harmful pattern. Adding a single conditional inside your typical 1200 line mixed-concern business-critical horrorshow function may assume a much greater liability (liability as in bug risk and liability as in risk/difficulty of future changes) than e.g. deleting a bunch of unused branches, or doing a function-extraction refactor pass. Standard "change one thing at a time" good engineering practices still apply of course.

memexy 6y ago

> Re-introduce duplication by inlining the abstracted code back into every caller. Ideally this type of workflow would be supported by the code editor. I've done this manually a few times and it's not fun.

nbardy 6y ago

This has been one of the hardest fought lessons I’ve learned it my programming career, but also one of the most fruitful. I am to make my abstractions too late rather than too early. My rule of thumb tends to me copy things six to seven times before you try to build an abstraction for it.

SkyPuncher 6y ago

I think there's a big cultural challenge with adopting duplication. It goes against most people's career growth objectives. Being able to effectively create clean, re-usable abstractions is a measure of being a "senior" engineer at many places. In other words, to be viewed as senior, you need to be able to effectively write abstraction frequently. It's hard to measure an abstraction in the moment, so a lot of people assume that the senior simply knows better. I find this extends to a lot of programming. Seniors will often use unnecessary tricks or paradigms simply because they can. It can make it extremely difficult for junior developers to grok code. Often this re-enforces seniority. "If only the seniors can work on a section of code, then they are senior". Likewise, there are so many books on crazy architectures and patterns. It's really neat to understand, but I've determined those books are pretty much self-serving. ---- I've found that my work is often far more limited by the domain/business logic than any sort of programming logic. I'll happily write code that looks really basic - because I know ANYBODY can come in and work with that code. If I write code that a junior needs to ask me questions like "what is this pattern?" or "what does this mean?", I've written bad code. ----- With all that being said, every single job interview I've ever had expects me to write code at the level of complexity that my title will be at. They'd much rather see me build some sort of abstract/brittle concept than using some constants and switch statements. The prior looks cool, the latter looks normal.

leto_ii 6y ago
> I think there's a big cultural challenge with adopting duplication. It goes against most people's career growth objectives. My experience is the complete opposite :D. What I've noticed is that the people who 'deliver' quickly (without much regard for what might be called code quality) and fulfill business requirements without much questioning are perceived as more valuable. > I've found that my work is often far more limited by the domain/business logic than any sort of programming logic. I broadly agree with this statement. However, just like a good carpenter knows how to properly build a bookcase, a table, a roof etc. a good developer should understand the programming logic and know how to apply it. Business requirements need to be fulfilled, but it's up to us to decide how to do that. More so, I think it's up to us to push back when we feel business requirements don't make sense from a technical point of view, or even from a business point of view.

cjfd 6y ago

If there is one single article about programming that I hate it is this one. It is completely the wrong message. One should instead be very eager to eliminate duplication. To avoid the pitfalls that the article notes one should create abstractions that are the minimal ones required to remove the duplication to avoid over-engineering. Also one should keep improving the abstractions. That way one can turn the abstraction that turned out to be wrong into the right one. It is the attitude of constant improvement that will make one succeed as opposed to the attitude of fear of changing something that this article seems to encourage. When one does things one learns. When one is afraid to try things everything will just calcify until it is no longer possible to add any new features. What one does need to make the refactoring work is automated tests.

Ensorceled 6y ago
In 30 years, I can count on the fingers of one hand the number of times I've encountered projects that were in trouble because there was copy/pasted code everywhere and the team was not abstracting out of fear of breaking the existing code. What I have encountered is dozens of projects that had essentially ground to a halt because of numerous deeply, and incorrectly, abstracted systems, modules and libraries. Correcting projects in this state has almost always been refactoring into fewer abstractions; less complex, more cohesive and less coupling.
- dragonwriter 6y ago
  > In 30 years, I can count on the fingers of one hand the number of times I've encountered projects that were in trouble because there was copy/pasted code everywhere and the team was not abstracting out of fear of breaking the existing code. I think the level of experience where underabstraction is common as opposed to overabstraction is so low that it's uncommon to find a team where that gets through, because even if someone junior is at the level where it's common, they’ll get corrected before it becomes a widespread problem.
- zbentley 6y ago
  I don't disagree and have seen the same thing. However, I've also noticed in those cases that it's very hard to get people to agree on what the problem actually is. One person's incorrect over-abstraction is another person's incompletely-DRYed-up code.

allenu 6y ago

In a large organization, the other thing you notice with trying to fix duplicated code is, if you take on refactoring it all, you are now responsible to make sure everything still works AND that you do not inhibit any future work. You are now responsible for more than you may have bargained for. Coming up with the right abstraction takes some predicting of future use-cases. It's more than just refactoring work to put it all in one place.

thinkloop 6y ago

A related problem: duplication is not equality. If two things happen to be the same right now, it doesn't mean they are intrinsically the same thing. If you have multiple products selling for $59.99 they shouldn't share a function to generate the "duplicate" price. Abstractions needs to be driven by conceptual equivalence, not value equivalence, where duplication is a good hint for a potential candidate of abstraction, but not the complete answer alone.

tomphoolery 6y ago

This again?? ;) I love this post. A lot of wasted hours were spent in the past trying to use abstractions that no longer made sense, but Sandi encouraged me to go back and rethink a lot of that and now my code is way easier to read. Thanks Sandi!

kureikain 6y ago

I think one of the cool thing about pattern matching or language(In my case, it's Elixir) that support function operator is we can have same method with different argument sigunatures. So we don't have to duplicate or inherit whatever and still share some common method.

haolez 6y ago

That's mostly how I matured as a developer: I find myself abstracting less and writing less code today than I did 10 years ago, but I'm more productive today, my code is cheaper to maintain and has fewer bugs. Sometimes, I will literally copy paste a small amount of logic just to avoid making a future reader of this code to keep hunting around where the business logic is actually implemented. "It's right here, my dear future reader!". Or maybe I was just a really bad programmer 10 years go :)

kristo 6y ago

There should be a code tool to re-inline code from an abstraction

worik 6y ago

Really this is stating the obvious. The social problem at step 6, 7, and 8 is a social and economic one. Having the time, resources, and skill to do a job properly is very important. But there are social and economic pressures to "just get it done". This is a specific formulation of a general problem.

avodonosov 6y ago

> they alter the code to take a parameter, and then add logic to conditionally do the right thing based on the value of that parameter But that's a textbook example of bad code, competent coders don't do this. Update: for example see Thinking Forth chapter "Factoring Techniques", around the tip "Don’t pass control flags downward.". Page 174 in the onscreen PDF downloadable from sourceforge. And there is no need for duplication. The bigger function can be split into several parts so that instead of one call with flag everyone calls needed set of smaller functions.

dragonwriter 6y ago
Competent coders do suboptimal things all the time, especially when there is delivery pressure; competent doesn't mean infallible or perfect. There's also not a clear boundary between what is a single appropriate abstraction and two (or N) distinct but superficially related concepts.
zbentley 6y ago
> that's a textbook example of bad code, competent coders don't do this. That's reductive and dismissive. There's a ton of subtlety in even defining the terms for that "best practice". What counts as a control flag versus a necessary choice that must be made by callers? Are you still passing control flags if you combine them into a settings object? What if you use a builder pattern to configure flags before invoking the business logic--is that better/worse/the same? What if you capture settings inside a closure and pass that around as a callback? How far "downward" is too far? How far is not far enough (e.g. all callers are inlining every decision point)? The answer to all of those is, of course, "it depends on a lot of things". And that's before you even get into the reality (which a sibling comment pointed out) that even if we grant that this is inherently bad code, that doesn't imply anything about the competence of the coder--some folks aren't put in positions where they can do a good job. Unrelated aside: Thinking Forth is an excellent book! Easy to jump into/out of in a "bite size" way, applicable to all sorts of programming, not just Forth programming.

vxNsr 6y ago

I want to thank everyone here, I’ve been stuck for about a week now on an issue that is entirely germane to this topic and the whole conversation here really helped me flesh out what was wrong and allowed me to understand a path forward. I’m honestly holding myself back from popping onto my computer right now to start working on it.

gm 6y ago

This advice just _feels_ very wrong. After thinking about it and seeing the other comments, some remarks: 1) It's fine to go back and duplicate code after you correct the abstraction. But it should be the _first_ phase in doing a larger pass to refactor code to fit the current business requirements. If you forgo the _second_ step, which should be to search for suitable abstractions again, you are absolutely guaranteed to be left with shit code that breaks in this situation, but not that other one, and no one knows why. I would absolutely only duplicate code as the prequel to deduplicating it again with updated abstractions. 2) If you do any of this without thorough unit tests you're insane. Keep the wrongly-abstracted code unless you have time to thoroughly fix the mess you will have made when you duplicate code again and introduce bugs (you're human, after all). 2a) If you are going to do this and there are no unit tests, create those unit tests before you touch the code initially (before the duplication). 3) Some of the comments saying you should wait until you implement something two or three times before creating an abstraction seem like comp sci 101 rules of thumb. It's way too simplistic a rule, way too general. Prematurely abstracted (haha!). The type of project and the type of company/industry will tell you what the right tradeoff is. That is all.

haolez 6y ago
You are assuming that the code is a moving target. Not every software project behaves that way. Sometimes, the software gets done as is.
- gm 6y ago
  In that case, then the original problem (incorrect abstraction) does not exist, or at least does not get worse over time, and thus does not need fixing.
roryokane 6y ago
The article already agrees with you on point 1: > Once you completely remove the old abstraction you can start anew, re-isolating duplication and re-extracting abstractions.

nfw2 6y ago

When considering whether some abstraction is "right" or "wrong", another important thing to consider is how cleanly the abstraction fits into a mental model of how the program works. Good abstractions provide value outside of removing duplication. They help us reason about a program by providing compression of logical concepts. Consider some helper function: "convertSnakeToCamelCase." This abstraction would take a string, do some operations on it, and return another string. It is easy to understand what the input and output is without having to think about these operations. This abstraction provides a benefit for anyone having to think about the program because it reduces the amount of concepts the reader has to parse from N (where N is the number of operations) to 1. This is helpful because people have limited mental bandwidth and can only reason with a finite number of concepts at any given time. Consider a different helper function: “processDataPayload.” This function takes data in some arbitrary complex shape and returns data in some arbitrary complex shape. The abstraction effectively communicates nothing to the reader, and it is actively unhelpful because it forces that person to follow a reference, remember all the details of what that function does, and substitute those details into the original function. Trying to find the conceptual boundaries that make the program easiest to reason about IMO is more of an art than a science and difficult to govern with hard and fast rules.

jasonhansel 6y ago
Agreed. I also think it's important to create abstractions that provide guarantees and/or maintain invariants. That way, your abstractions actually help you be more confident that your code is correct. The point of abstraction isn't per se to reduce duplication--it's to make your code more straightforward and to make errors more obvious.

recroad 6y ago

Programmer B in Step 6 should have used SOLID and refactored to extend the module (or something similar). This is strawman argument which has little to do with the "wrong" abstraction and everything to do with poor design choices.

kuharich 6y ago

Prior comments: https://news.ycombinator.com/item?id=17578714 https://news.ycombinator.com/item?id=17578714

ragnese 6y ago

I can't help but wonder if we're sometimes using the wrong words for things. In this discussion we keep talking about "code duplication" and "abstraction" hand-in-hand, but I think they're almost orthogonal concepts, at least as I think of them. Seeing the same code almost copy-and-pasted in a few places might call for some code-deduplication. But that's not necessarily a new "abstraction" in my eyes. It may be, but it also may not be. I'm struggling to think of a specific example because I fully intended to go to bed before arriving here... But as a really stupid example, let's say you have `val a = x + x + x` and `val b = y + y + y` and `val c = z + z + z` in your code. If you write a new function like `fun addThreeTimes(i) = i + i + i`, I don't see that as a new abstraction at all. If, however, you invent multiplication, now you're at a new abstraction! `val a = x * 3; val b = y * 3`, etc. "Abstraction" to me is about thinking at a different semantic level, not about avoiding copy and paste. Does this resonate with anyone else? Am I missing the point?

lambdatronics 6y ago
I would say it becomes an abstraction when it needs parameters in addition to the nominal arguments. So in the case of "addThreeTimes(i)", sure, that's just abbreviation, but "addNtimes(i,N)" is a bona-fide abstraction. Edit: Reminds me of the distinction between inheritance and subtyping: https://www.cs.utexas.edu/users/wcook/papers/InheritanceSubtyping90/CookPOPL90.pdf https://www.cs.utexas.edu/users/wcook/papers/InheritanceSubt...
- ragnese 6y ago
  I agree with your point and your edit. The inheritance thing is very similar to what I'm talking about!
epage 6y ago
In my experience, code dedupping without shared requiremeots hurts you over time. Nothing is keeping the code focused so as requirements change, you either fork it or add "one more flag". Focusing on abstractions is what helps me in dedupping with a focus on requirents.
MaulingMonkey 6y ago

ricksharp 6y ago

The mistake is creating an abstraction because of seeing duplication. DRY is not a good guiding principle. It is an anti-principle. Abstractions should only be created when they have a clear purpose and create a simpler architecture by encapsulating a single concern. The reality is that all code is duplication. The reason we write code is because it is the most concise language to specify the intended goal in the current context. What is unique is not the code that we are writing. The unique part is the code in the current context and each level of abstraction separates the context from the implementation - so that abstraction must be beneficial in organizing the overall solution into individual logical components of singular concern.

runald 6y ago

For something that argues against bad abstractions, the article sure is lacking in concrete examples and makes a point in abstract. A lot of people will likely misinterpret or get the idea that abstraction is only done for duplicated code (DRY as some people would call it). I think the wrong/bad "abstractions" here mostly refers to abstraction that was made over common code that is very specific in a context and is very susceptible to domain changes. But there are a lot of other kinds of abstraction aside from DRY. There are abstractions made to reduce clutter and hide implementation detail and will likely be used only once. There are also abstraction that are more general and aren't coupled to the domain. These abstractions are more reusable and composable, and are immune to domain changes such as the step 6 in the article. Some people would find these kinds of abstractions harder to digest, but I personally consider these kinds of abstractions as extensions to the standard library, or even additions to the vocabulary of the programming language. Note that I don't claim that general abstractions are necessarily better, since the generality can be made to the extreme and we'd have monads for breakfast. All in all, I agree with the article, except that it is only referring to one kind of abstraction, although I hesitate to call it as such.

ninetax 6y ago

What are people's recommendations on books on how and when to create the right abstractions? Last year I read Zach Tellman's _Elements of Clojure_ and really loved the parts that touched on the subject of abstraction.

neetrain 6y ago

I think the term "wrong" causes all the misunderstandings. It sounds like the abstraction was wrong in the first place. Can it be called "rotten" abstraction?