12 ms·

The Wrong Abstraction

220 points by rumcajz 11y ago

rumcajz 11y ago
I wrote about the problem here: http://250bpm.com/blog:36 http://250bpm.com/blog:36 The fact that abstractions can often add complexity rather than remove it should be taught in schools. The problem with distinguishing between the two cases is that it often requires business domain knowledge. And programmers, sadly, rarely care about the domain logic.
- TeMPOraL 11y ago
  > And programmers, sadly, rarely care about the domain logic. Oh, but we do. It's the one thing we want to know but we pretty much never get from our customers. No surprises there - most people are not equipped to express their "domain logic" in a way that is useful for automating it; quite often they don't even understand it. Humans can hand-wave their way through anything, relying on intuition, patterns and methodologies. They don't need to understand what they're doing deep enough to express it as code. And so what usually happens is that your client/manager gives you a nonsense spec saying "this thing should do that when clicked, and that thing should do something else". And, as a programmer, you then have to reverse-engineer the domain abstractions from it; abstractions the people giving you the spec probably don't understand themselves.
  - rumcajz 11y ago
    What I meant is that looking at an average programmer's resume feels relly weird: A web shop. High-performance trading. Game development. Scientific computing. Telecom. And so on. No way anyone can be a domain expert it that many areas. Also, you often hear folks saying things like "I am a Java programmer." Stuff like "I am a telco programmer" is much more rare.
  - mbrock 11y ago
    That's why understanding the domain logic requires cooperation between programmers and those who understand the domain. It might not be that they don't understand the domain "deeply enough." Maybe they understand it in the way that is relevant for their work, which typically doesn't require formalization. Making sure this collaboration actually happens is one of the big topics of Domain-Driven Design. We can't just expect to be given a correct model; teasing that out is part of our job. If the spec is nonsense, we should say that and try to fix the process.
- mannykannot 11y ago
  Being able to lead your customers to explain clearly what they really want (and, in some cases, to help them understand why they can't have/afford it) is one of the skills of a productive programmer. It involves a lot more than just listening, though that is a good starting point.
mbrock 11y ago
Richard P. Gabriel wrote about this in connection with the idea of design patterns, e.g. in Patterns of Software. Here's a nice quote from the beginning of a chapter: > "The room fills with cold, conditioned air; outside the heat hazes, filtered through greened glass windows: a new building hardly first populated. The speaker is wild-eyed, explaining new ideas like a Bible thumper. His hair is a flat-top; his mouth frowns in near grimace. He strides to my seat, looks down and says in a Texas drawl, 'and the key is simply this: Abstractions. New and better abstractions. With them we can solve all our programming problems.'"
lmm 11y ago
This may be obvious, but when working on Java I've found that this applies not just to classes but also to methods. When refactoring a poorly-factored class, often the most effective way forward is to inline all the private methods (introducing a lot of duplication) and then factor out the duplication.
- mannykannot 11y ago
  It is even more effective when you are faced with a bunch of classes, most of which do nothing but delegate their responsibilities to others. I have a case in front of me right now. Data-flow analysis was a method that was swept away (somewhat unfairly) on the grounds of being too procedural and not OO. One of its key tenets was that once you had discovered the essential flows and interactions, you should discard the problem-space partitioning that helped you find the graph, and use the graph to partition the solution space (which might well result in the same partitioning, but not always.)
mannykannot 11y ago
While I have no objection to the notion that duplication can be the right choice in some circumstances, and I completely agree with the author in thinking that pressing on rather than backtracking and taking a different path is a source of much complexity, his pseudo-example misses a third alternative: instead of (re)introducing duplication wherever the current abstraction is used, derive a generic abstraction covering the common aspects of the previous and new requirements, and use that in implementing two new abstractions, one to replace the old, and the other for the new cases.
adajos 11y ago
If only Programmer B had avoided that initial choice to preserve the broken abstraction rather than abandoning it then and there. He must have been practicing Deadline Driven Development.
- protomyth 11y ago
  > He must have been practicing Deadline Driven Development. or The Team Has A Hammer Development
akud 11y ago
The problem was programmers between B and X didn't refactor to update the abstraction with their new understanding of the domain, and to better meet likely business requirements. When programmer A introduces an abstraction, it's not as if they are saying "this abstraction is the one true way to represent the domain." The abstraction is a tool. It helps cover some duplication, and ideally express intent. But the tool needs to be updated when it's not useful.
andreasklinger 11y ago
In my team we usually work with the principle of 3 "if it's non obvious wait for 3 cases before abstracting it"
tejohnso 11y ago
"Programmer B feels honor-bound to retain the existing abstraction" I think it's more often likely that the next person comes along and lazily forces the most brittle, minimal change possible to make their new requirement work without thinking about the larger context. Even if that means putting a complicated conditional into the existing function, while leaving the existing function name intact. This isn't about honor. It's simply bad programmer behavior. It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute.
- gingerlime 11y ago
  I agree. I think Sandi was just being generous and forgiving, for the sake of not getting sidetracked. The fact of the matter though is that those "small" refactors do take place quite commonly. Out of laziness or else, and start a slow spiral...
- amelius 11y ago
  > This isn't about honor. It's simply bad programmer behavior. It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute. Time pressure may also be an issue. Or, what if the system is only for prototyping, and later turns out to be needed for production (a management decision)?
  - lmm 11y ago
    Then management should be made aware that they're incurring technical debt by taking that decision. It can be worth compromising code quality for short-term goals on occasion, but you should do it with your eyes open.
- hoorayimhelping 11y ago
  >It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute. I think you might be committing the fundamental attribution error. )https://en.wikipedia.org/wiki/Fundamental_attribution_error https://en.wikipedia.org/wiki/Fundamental_attribution_error >the tendency for people to place an undue emphasis on internal characteristics (personality) to explain someone else's behavior in a given situation rather than considering the situation's external factors. Rather than considering external issues like time crunch, management pressure, or too-high a workload, or anything else it's much easier to just assume everyone else is a lazy and shitty programmer.
mojuba 11y ago
Or, in terms of OOP, wrong abstraction is when you have subclasses with pretty much everything redefined in each. I don't think this is very common though. You will more likely see a lot of duplication in poor legacy code, and not because the coders were so clever, but quite the opposite: it's when they barely even understand how to extract abstractions.
- arethuza 11y ago
  Or baroque structures of abstractions where you have a real problem finding where stuff actually gets done - sometimes combined with a need to base all application classes on a common shared abstract base class even though they really don't share anything.
JDDunn9 11y ago
A 3rd option would be to simply apply good programming standards like a function doing one thing and only one thing.
- rco8786 11y ago
  At some point, you're gonna have to introduce some logic into your program, and that logic is gonna have to be in a function.
wellpast 11y ago
I can tell an amateur programmer from a professional by looking at their order of priorities when they grow a code base. Amateur programmers tend to put code de-duplication at the top of their priority list and will burn the whole house down to that often-trivial end. This writer is pointing out that there are other concerns that far, far trump duplicated code -- and she's right. However she's not elaborating enough on what it is a "wrong abstraction." We can be more precise. The real offense when we factor duplicated code is the new dependency that is added to the system. And this is what amateurs don't understand. Every dependency you bring into your code architecture costs you and should be judiciously introduced. De-duplication of code ALONE is rarely a strong enough reason to add a dependency. If you want to be a professional programmer one of the most important things to acquire is a distaste for dependencies. Every dependency you add should be carefully considered (and lamented as a reluctant necessity if you decide to introduce it). As a 20 year veteran in this industry having worked on myriad code bases, I will always prefer a code base with duplicated code and fewer dependencies than the other way around. So back to the "wrong abstraction". When we compose systems, we are looking for fewest dependencies and stable dependencies. What I think the writer means by "the wrong abstraction" is a "volatile dependency". I'm trying to be precise here because a common reaction to terms like "the wrong abstraction" is that wrong/right, its all subjective. The truth of the matter is that it's not subjective at all -- the higher-quality system is the one with optimally few dependencies and stable dependencies, these are measurable qualities.
- dustingetz 11y ago
  The distaste is for complexity. Adding new functions to reduce duplication can add complexity. It can also reduce complexity. It can also decrease complexity for a while and then in the long run increase complexity. Recall the famous quote, "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." Sometimes dependencies are really cheap - when they are the right abstraction. I think the article put it fine. Rails itself is an example of an abstraction that reduces complexity for a while and then adds complexity when you reach a certain size. So it was the right abstraction at first, and then the requirements change and it slips over to the wrong abstraction. Here is an insightful comment about why that is both inevitable and doesn't matter: https://news.ycombinator.com/item?id=11028885 https://news.ycombinator.com/item?id=11028885
mattiemass 11y ago
It's taken me perhaps longer that t should have to get here, but boy oh boy do I think this is spot on.
chowells 11y ago
If you found yourself agreeing with article, yet have ever mocked mathematically-derived abstraction patterns like monads, it's time to reconsider. Math is, at its core, the study of abstraction. Things that have been found to be good abstractions in math probably are good abstractions in programming. Learn from history. Mathematically-derived abstractions probably are used because they are the right abstractions.
- spion 11y ago
  Agreed, but I think the reason for the mocking is that they're notoriously hard to understand being several layers removed from the programming context. I understand monads in the context of programming, but I wouldn't pretend to understand how they could be applied in other fields. Haven't got the slightest idea. Perhaps importing abstractions straight from mathematics isn't the only answer. We might do better if we simply added some type and law restrictions to our abstractions as well as clearly stated assumptions. That way everyone would know when they don't apply anymore.
  - such_a_casual 11y ago
    Programming languages are some of the shittiest languages ever created. Mathematics is probably the best language ever created. The more programming can be like math, the better.
- lackbeard 11y ago
  I don't think math is the study of abstraction. It seems to me it's proving truths about formal systems. Abstraction has the same purpose in math as it does in programming, a tool to more effectively communicate ideas.
  - sebastos 11y ago
    Not sure I agree. Usually, mathematics is developed backwards - you have a concrete question that you want to answer, and you reason that it can be answered so long as such and such as true. The formal system is developed post-hoc to give yourself a language to reason in, but you're really trying to take the result you already "knew" to be true, and find the least restrictive system description to which it still applies. Then you look for parallels and generalizations. The abstractions are in a large sense the most important and difficult thing to create, because you're creating a schematic and saying "if you can rephrase your problem into these terms, then I already proved this intuitively correct thing for you, so you don't have to worry about edge cases". Algebra, geometry, calculus, etc. all follow this model. A lot of the conclusions in, e.g. analysis are obvious once you impose continuity. The object of mathematics is to produce abstractions that make proofs possible or trivial. Sets, fields, groups, categories, functions, integers, reals, complex numbers, quaternions - these aren't notational conveniences that are introduced to make it easier to talk about. They're important because if you have something with the properties of a group, you know some powerful truths about it for free. If you change anything about the definition of group, the set of truths you know changes. People have been grinding on what the definition of a set should be for a century, trying to build the best possible abstraction, and they have all of the same problems coders have. You assume to much, it's not very general. Don't assume enough, there's nothing interesting to say that's unilaterally true. So I would argue that characterizing math as the study of abstraction is largely fair. You "see" a result is probably true some of the time, and then you try to find out just how general you can make that statement.
jarpineh 11y ago
I read somewhere, but have forgotten where, that having doing things same way twice is ok, only when you're second time repeating yourself, think about using an abstraction. Or to say it more succintly, duplication is not a problem, triplication is ;) I have found that following this simple rule I allow the development to go forward, and can go back to it when I have better knowledge of the system at large. At later date I might have a better luck in choosing the correct abstraction.
amelius 11y ago
You: "We can implement this new feature but doing it right would mean we have to design a new architecture for the entire system, and that would take several months. Or, we could just hack it, change two lines of code and be done with it, but it might bite us in the future." Management: "Change those two lines of code."
ska 11y ago
On the other hand, moving to the right abstraction is one of the most powerful ways to improve a code base. Often the right abstraction cannot easily be seen from existing code - they come from the domain.
- kaeluka 11y ago
  Someone once told me: "Just looking at Java code, you'd never find monads."
tempodox 11y ago
There is another koan I would recommend to the interested: Quantity is easier to control than complexity. I propose that “the wrong abstraction” is “complexity without benefits”. In light of the sentence above, duplication is clearly the lesser evil.
nine_k 11y ago
TL;DR: If you have made a mistake, return back to the previous known working state, proceed from there. It does not matter much which was the specific mistake, really: a bad choice of an abstraction or something else. Good abstractions help immensely; programming is all about them. Let's not forget it, too.
bograt 11y ago
My own observation is that programmers have a marked tendency to assume that because two methods have similar or identical behaviour, that this implies they should share an implementation. In large code bases, its common to find methods that have identical implementations 'by chance'. By which I mean that the commonality is a side effect of requirements that could readily change. In my opinion, the best defence against this is good documentation: if a two methods have clearly documented behaviours, then even if their implementations have been fused, a subsequent programmer will have more context (and more confidence) reduplicating the code in response to further changes.
deleted 11y ago
[deleted]
agentultra 11y ago
I hate needless abstractions for the sake of removing duplication, DRY, etc. It's mad. Most of these languages don't have proper macros and so you end up going through these logical contortions to achieve a particular pattern of behavior... and it's just noise. Anyone who comes along can look at that code for hours and never know whether it performs any work (or even what that work is). A little duplication is fine until you figure out what the real problem is. The problem is usually in your data. Maybe it's not structured properly or you need to simplify the steps to transform it at an earlier stage. A good specification will go a long way to reducing the desire to introduce hapless "abstractions." An abstraction, in the mathematical sense, will hold over the domain and introducing a new one merely allows you to manipulate objects on the lower level using new algebras, predicates, etc. Code abstractions are often the leakiest abstractions. Especially the kind the author is talking about. Avoid them. Even if you have a little bit of duplication. Only start worrying about that duplication when it starts spanning compilation units/modules/whatever and is actually causing problems. Then look at the data and figure out how to structure it so you don't need that code all over the place.
- Chris2048 11y ago
  I think lack of things like macros is the bigger problem. There are a limited ways to abstract, or represent duplications, in a lot of languages. Sometime a comment in an issue tracker might help.
protomyth 11y ago
Isn't a bit of this governed by which programming paradigm your language uses? I get the feeling the answer and approach are much different when comparing Forth to Java.
- drostie 11y ago
  Yes. In theory there is a programming paradigm which fixes this, called "aspect oriented programming." I haven't seen a really accessible aspect-oriented system however.
  - protomyth 11y ago
    patents will do that My point was that the way your language builds abstractions might increase or decrease the problem.
  - collyw 11y ago
    Decorators in Python are pretty much the same idea. I see them used quite a lot in Flask and Django.
dandare 11y ago
Firstly, there is no such things as perfect abstractions, hence the The Law of Leaky Abstractions. Secondly, without abstraction we would be writing in machine code. So unless the authors enlightens us on what makes an abstraction wrong the whole discussion is a waste of time.
dreamdu5t 11y ago
False dichotomy. You can avoid code duplications while using the "right" abstractions.
- hnbrox 11y ago
  expand?
_pmf_ 11y ago
The same applies to dependencies. I've grown very reluctant towards introducing a dependency for the sake of deduplication. I'd rather have isolated modules than a de-duplicated Big Ball of Mud.
grandalf 11y ago
It's funny how often I've made this argument with even fairly experienced programmers and they seem to have a visceral reaction to the code being "less dry" than it could possibly be. Similarly, once a codebase uses several very wrong abstractions, it becomes significantly more confusing to work on, exponentially increasing cost. The temptation to use a mature library as a dependency is very strong since time is initially saved. When that library introduces the wrong abstraction, the consequences can be severe. I typically argue for (at least) creating the correct abstraction and having it wrap the mature library to constrain or properly name its behavior, and to make it less strongly coupled to the rest of the system. It doesn't just take experience to realize these sorts of things, it takes a willingness to question one's own code and imagine how it might have been done better, or how it might appear to someone who didn't write it.
- TheOtherHobbes 11y ago
  It's all a bit no-true-Scotsman though. abstraction != dry bad abstraction != anti-dry Abstraction is primarily about separation of concerns, not about avoiding repetition. Drying out code that's repeated all over isn't the same as creating a formal abstraction for some element of the overall logic. Which is why >once a codebase uses several very wrong abstractions, it becomes significantly more confusing to work on, exponentially increasing cost. And drying out code makes it easier to maintain, but it doesn't guarantee that the architecture isn't a mess. The problem is perhaps that CS teaches algos, and sometimes it teaches design patterns. But there's almost no useful theory of abstraction design. Design patterns are more or less as good as it gets, and all they do is give give you a cookbook of stock formulas to try. Beyond that, there's no useful way to reason about abstractions, test them for domain fit, or rate them for elegance and efficiency.
  - grandalf 11y ago
    Precisely. Very well put. If anything, some developers use design patterns as a grab bag when solving a problem... a better approach is to model the solution and then be ready to "back in" to a design pattern upon noticing strong similarity or observing that the design pattern is a bit more abstract way of doing the same thing. Because of the tendency to pick a pattern first and design for the domain later, many instances of design pattern use in the wild are subtly (or not so subtly) incorrect. > Beyond that, there's no useful way to reason about abstractions, test them for domain fit, or rate them for elegance and efficiency. Very true indeed. Looking at a design through the lens of coupling and testability and modularity is a good start and can reveal many problems, but I think the real gotcha has to do with naming: Once we name a concept/abstraction we are likely to reason within that abstraction, and we rarely consider whether we are stretching it too far, or if it is even that thing anymore.
cpeterso 11y ago
I'm reminded off ESR's "curse of the gifted" email, schooling Linus Torvalds on the lack of modularization and code-sharing in Linux's driver code. :) http://lwn.net/2000/0824/a/esr-sharing.php3 http://lwn.net/2000/0824/a/esr-sharing.php3
- GFK_of_xmaspast 11y ago
  I'm reminded of the "I'm with those guys" comic.
- hinkley 11y ago
  Oh my. I recall reading this years ago. I had totally forgotten that Eric basically tells Linus to grow up and learn to use version control. Now we're all using Linus' version control.
Almaviva 11y ago
For many decisions in programming, there are tradeoffs, and relying on general rules of thumb can be much worse than judging the specific case. And heuristics are often interpreted differently by different people. Such is this discussion, imo. It's very possible to over-abstract. If something isn't extended, maybe it shouldn't be designed to be extendable, and if it isn't configured, maybe it shouldn't be configurable. On the other hand, abstractions are the heart of programming, and finding a good one is what can make you much more productive and maybe even have more fun. The feeling when I've abstracted something and got more out than I put in is maybe the best feeling there is in this craft. The difficulty is in finding the balance. I also dislike the rule of 3 as a hard general rule. There are many times when it makes sense to abstract something out of 2 uses, if there is a lot of code and clear separation and commonality involved, or you anticipate more uses later. Maybe even abstracting sub-problems out of one case makes sense, if separating the problem cleanly into two parts makes it easier to reason about. There are many times when it's a bad idea too; it depends on the specifics and it's a judgement call based on experience. (And, I would even say, there is room for personal taste in programming about this; where the sweet spot lies may vary from person to person.)
guscost 11y ago
One thing I've put a lot of thought into recently is what parts of a program should typically be abstracted, because when you come across them they might be your best bets for big wins. Top of the list: constants should probably all go in special structures with the goal of guaranteeing consistency and making them easy to understand at a glance. Everyone might know that charge code "X" means a check payment, but what happens when a new developer looks at that code with "X"s everywhere? It's more verbose to use constants.chargeCodes.CHECK_PAYMENT but nobody will misunderstand what you mean, and your IDE will be able to verify that your codes are valid. That's worth an awful lot of extra characters. Bonus win: when your legacy backend finally gets upgraded, you have the option to change to a new code for check payments, easy as pie.
hex13 11y ago
This is the old truth - premature abstraction is a root of all evil (I know original quote was "optimization", but isn't an abstraction just some "architectural optimization"? ;) On the other hand: I often see copy pasted code which have these same characteristics author wrote about: "Another additional parameter. Another new conditional. Loop until code becomes incomprehensible." If somebody writes too many conditionals, parameters and creates incomprehensible loops, then he will do it always, no matter whether in abstracted code or in copy pasted code... The problem is that most of programmers are not skilled enough to write good code. So sometimes there is a need for refactor wrong abstractions in legacy code, sometimes there is need for cleaning tons of copy-pasted code and make good abstractions... Either way - maintaining is hard.
daxfohl 11y ago
Meh, it's an easy situation to get out of though. Especially with a typed language and decent IDE: right-click, "inline function"; check your git status to see what changed, go to those files and remove unused branches (which a decent IDE should hightlight and provide 2-keystroke fixes for). 2 minutes, done. Grotesquely replicated code is a much more difficult situation to resolve. I'm guessing the OP primarily uses untyped languages. I'd offer that that is the core of the problem, not the abstractions.

220 points by rumcajz 11y ago

I wrote about the problem here: http://250bpm.com/blog:36 http://250bpm.com/blog:36 The fact that abstractions can often add complexity rather than remove it should be taught in schools. The problem with distinguishing between the two cases is that it often requires business domain knowledge. And programmers, sadly, rarely care about the domain logic.

TeMPOraL 11y ago
> And programmers, sadly, rarely care about the domain logic. Oh, but we do. It's the one thing we want to know but we pretty much never get from our customers. No surprises there - most people are not equipped to express their "domain logic" in a way that is useful for automating it; quite often they don't even understand it. Humans can hand-wave their way through anything, relying on intuition, patterns and methodologies. They don't need to understand what they're doing deep enough to express it as code. And so what usually happens is that your client/manager gives you a nonsense spec saying "this thing should do that when clicked, and that thing should do something else". And, as a programmer, you then have to reverse-engineer the domain abstractions from it; abstractions the people giving you the spec probably don't understand themselves.
- rumcajz 11y ago
  What I meant is that looking at an average programmer's resume feels relly weird: A web shop. High-performance trading. Game development. Scientific computing. Telecom. And so on. No way anyone can be a domain expert it that many areas. Also, you often hear folks saying things like "I am a Java programmer." Stuff like "I am a telco programmer" is much more rare.
- mbrock 11y ago
  That's why understanding the domain logic requires cooperation between programmers and those who understand the domain. It might not be that they don't understand the domain "deeply enough." Maybe they understand it in the way that is relevant for their work, which typically doesn't require formalization. Making sure this collaboration actually happens is one of the big topics of Domain-Driven Design. We can't just expect to be given a correct model; teasing that out is part of our job. If the spec is nonsense, we should say that and try to fix the process.
mannykannot 11y ago
Being able to lead your customers to explain clearly what they really want (and, in some cases, to help them understand why they can't have/afford it) is one of the skills of a productive programmer. It involves a lot more than just listening, though that is a good starting point.

mbrock 11y ago

Richard P. Gabriel wrote about this in connection with the idea of design patterns, e.g. in Patterns of Software. Here's a nice quote from the beginning of a chapter: > "The room fills with cold, conditioned air; outside the heat hazes, filtered through greened glass windows: a new building hardly first populated. The speaker is wild-eyed, explaining new ideas like a Bible thumper. His hair is a flat-top; his mouth frowns in near grimace. He strides to my seat, looks down and says in a Texas drawl, 'and the key is simply this: Abstractions. New and better abstractions. With them we can solve all our programming problems.'"

lmm 11y ago

This may be obvious, but when working on Java I've found that this applies not just to classes but also to methods. When refactoring a poorly-factored class, often the most effective way forward is to inline all the private methods (introducing a lot of duplication) and then factor out the duplication.

mannykannot 11y ago
It is even more effective when you are faced with a bunch of classes, most of which do nothing but delegate their responsibilities to others. I have a case in front of me right now. Data-flow analysis was a method that was swept away (somewhat unfairly) on the grounds of being too procedural and not OO. One of its key tenets was that once you had discovered the essential flows and interactions, you should discard the problem-space partitioning that helped you find the graph, and use the graph to partition the solution space (which might well result in the same partitioning, but not always.)

mannykannot 11y ago

While I have no objection to the notion that duplication can be the right choice in some circumstances, and I completely agree with the author in thinking that pressing on rather than backtracking and taking a different path is a source of much complexity, his pseudo-example misses a third alternative: instead of (re)introducing duplication wherever the current abstraction is used, derive a generic abstraction covering the common aspects of the previous and new requirements, and use that in implementing two new abstractions, one to replace the old, and the other for the new cases.

adajos 11y ago

If only Programmer B had avoided that initial choice to preserve the broken abstraction rather than abandoning it then and there. He must have been practicing Deadline Driven Development.

protomyth 11y ago
> He must have been practicing Deadline Driven Development. or The Team Has A Hammer Development

akud 11y ago

The problem was programmers between B and X didn't refactor to update the abstraction with their new understanding of the domain, and to better meet likely business requirements. When programmer A introduces an abstraction, it's not as if they are saying "this abstraction is the one true way to represent the domain." The abstraction is a tool. It helps cover some duplication, and ideally express intent. But the tool needs to be updated when it's not useful.

andreasklinger 11y ago

In my team we usually work with the principle of 3 "if it's non obvious wait for 3 cases before abstracting it"

tejohnso 11y ago

"Programmer B feels honor-bound to retain the existing abstraction" I think it's more often likely that the next person comes along and lazily forces the most brittle, minimal change possible to make their new requirement work without thinking about the larger context. Even if that means putting a complicated conditional into the existing function, while leaving the existing function name intact. This isn't about honor. It's simply bad programmer behavior. It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute.

gingerlime 11y ago
I agree. I think Sandi was just being generous and forgiving, for the sake of not getting sidetracked. The fact of the matter though is that those "small" refactors do take place quite commonly. Out of laziness or else, and start a slow spiral...
amelius 11y ago
> This isn't about honor. It's simply bad programmer behavior. It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute. Time pressure may also be an issue. Or, what if the system is only for prototyping, and later turns out to be needed for production (a management decision)?
- lmm 11y ago
  Then management should be made aware that they're incurring technical debt by taking that decision. It can be worth compromising code quality for short-term goals on occasion, but you should do it with your eyes open.
hoorayimhelping 11y ago
>It might be due to lack of skill or experience, or maybe laziness, or lack of discipline, or some other negative attribute. I think you might be committing the fundamental attribution error. )https://en.wikipedia.org/wiki/Fundamental_attribution_error https://en.wikipedia.org/wiki/Fundamental_attribution_error >the tendency for people to place an undue emphasis on internal characteristics (personality) to explain someone else's behavior in a given situation rather than considering the situation's external factors. Rather than considering external issues like time crunch, management pressure, or too-high a workload, or anything else it's much easier to just assume everyone else is a lazy and shitty programmer.

mojuba 11y ago

Or, in terms of OOP, wrong abstraction is when you have subclasses with pretty much everything redefined in each. I don't think this is very common though. You will more likely see a lot of duplication in poor legacy code, and not because the coders were so clever, but quite the opposite: it's when they barely even understand how to extract abstractions.

arethuza 11y ago
Or baroque structures of abstractions where you have a real problem finding where stuff actually gets done - sometimes combined with a need to base all application classes on a common shared abstract base class even though they really don't share anything.

JDDunn9 11y ago

A 3rd option would be to simply apply good programming standards like a function doing one thing and only one thing.

rco8786 11y ago
At some point, you're gonna have to introduce some logic into your program, and that logic is gonna have to be in a function.

wellpast 11y ago

I can tell an amateur programmer from a professional by looking at their order of priorities when they grow a code base. Amateur programmers tend to put code de-duplication at the top of their priority list and will burn the whole house down to that often-trivial end. This writer is pointing out that there are other concerns that far, far trump duplicated code -- and she's right. However she's not elaborating enough on what it is a "wrong abstraction." We can be more precise. The real offense when we factor duplicated code is the new dependency that is added to the system. And this is what amateurs don't understand. Every dependency you bring into your code architecture costs you and should be judiciously introduced. De-duplication of code ALONE is rarely a strong enough reason to add a dependency. If you want to be a professional programmer one of the most important things to acquire is a distaste for dependencies. Every dependency you add should be carefully considered (and lamented as a reluctant necessity if you decide to introduce it). As a 20 year veteran in this industry having worked on myriad code bases, I will always prefer a code base with duplicated code and fewer dependencies than the other way around. So back to the "wrong abstraction". When we compose systems, we are looking for fewest dependencies and stable dependencies. What I think the writer means by "the wrong abstraction" is a "volatile dependency". I'm trying to be precise here because a common reaction to terms like "the wrong abstraction" is that wrong/right, its all subjective. The truth of the matter is that it's not subjective at all -- the higher-quality system is the one with optimally few dependencies and stable dependencies, these are measurable qualities.

dustingetz 11y ago
The distaste is for complexity. Adding new functions to reduce duplication can add complexity. It can also reduce complexity. It can also decrease complexity for a while and then in the long run increase complexity. Recall the famous quote, "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." Sometimes dependencies are really cheap - when they are the right abstraction. I think the article put it fine. Rails itself is an example of an abstraction that reduces complexity for a while and then adds complexity when you reach a certain size. So it was the right abstraction at first, and then the requirements change and it slips over to the wrong abstraction. Here is an insightful comment about why that is both inevitable and doesn't matter: https://news.ycombinator.com/item?id=11028885 https://news.ycombinator.com/item?id=11028885

mattiemass 11y ago

It's taken me perhaps longer that t should have to get here, but boy oh boy do I think this is spot on.

chowells 11y ago

If you found yourself agreeing with article, yet have ever mocked mathematically-derived abstraction patterns like monads, it's time to reconsider. Math is, at its core, the study of abstraction. Things that have been found to be good abstractions in math probably are good abstractions in programming. Learn from history. Mathematically-derived abstractions probably are used because they are the right abstractions.

spion 11y ago
Agreed, but I think the reason for the mocking is that they're notoriously hard to understand being several layers removed from the programming context. I understand monads in the context of programming, but I wouldn't pretend to understand how they could be applied in other fields. Haven't got the slightest idea. Perhaps importing abstractions straight from mathematics isn't the only answer. We might do better if we simply added some type and law restrictions to our abstractions as well as clearly stated assumptions. That way everyone would know when they don't apply anymore.
- such_a_casual 11y ago
  Programming languages are some of the shittiest languages ever created. Mathematics is probably the best language ever created. The more programming can be like math, the better.
lackbeard 11y ago
I don't think math is the study of abstraction. It seems to me it's proving truths about formal systems. Abstraction has the same purpose in math as it does in programming, a tool to more effectively communicate ideas.
- sebastos 11y ago
  Not sure I agree. Usually, mathematics is developed backwards - you have a concrete question that you want to answer, and you reason that it can be answered so long as such and such as true. The formal system is developed post-hoc to give yourself a language to reason in, but you're really trying to take the result you already "knew" to be true, and find the least restrictive system description to which it still applies. Then you look for parallels and generalizations. The abstractions are in a large sense the most important and difficult thing to create, because you're creating a schematic and saying "if you can rephrase your problem into these terms, then I already proved this intuitively correct thing for you, so you don't have to worry about edge cases". Algebra, geometry, calculus, etc. all follow this model. A lot of the conclusions in, e.g. analysis are obvious once you impose continuity. The object of mathematics is to produce abstractions that make proofs possible or trivial. Sets, fields, groups, categories, functions, integers, reals, complex numbers, quaternions - these aren't notational conveniences that are introduced to make it easier to talk about. They're important because if you have something with the properties of a group, you know some powerful truths about it for free. If you change anything about the definition of group, the set of truths you know changes. People have been grinding on what the definition of a set should be for a century, trying to build the best possible abstraction, and they have all of the same problems coders have. You assume to much, it's not very general. Don't assume enough, there's nothing interesting to say that's unilaterally true. So I would argue that characterizing math as the study of abstraction is largely fair. You "see" a result is probably true some of the time, and then you try to find out just how general you can make that statement.

jarpineh 11y ago

I read somewhere, but have forgotten where, that having doing things same way twice is ok, only when you're second time repeating yourself, think about using an abstraction. Or to say it more succintly, duplication is not a problem, triplication is ;) I have found that following this simple rule I allow the development to go forward, and can go back to it when I have better knowledge of the system at large. At later date I might have a better luck in choosing the correct abstraction.

amelius 11y ago

You: "We can implement this new feature but doing it right would mean we have to design a new architecture for the entire system, and that would take several months. Or, we could just hack it, change two lines of code and be done with it, but it might bite us in the future." Management: "Change those two lines of code."

ska 11y ago

On the other hand, moving to the right abstraction is one of the most powerful ways to improve a code base. Often the right abstraction cannot easily be seen from existing code - they come from the domain.

kaeluka 11y ago
Someone once told me: "Just looking at Java code, you'd never find monads."

tempodox 11y ago

There is another koan I would recommend to the interested: Quantity is easier to control than complexity. I propose that “the wrong abstraction” is “complexity without benefits”. In light of the sentence above, duplication is clearly the lesser evil.

nine_k 11y ago

TL;DR: If you have made a mistake, return back to the previous known working state, proceed from there. It does not matter much which was the specific mistake, really: a bad choice of an abstraction or something else. Good abstractions help immensely; programming is all about them. Let's not forget it, too.

bograt 11y ago

My own observation is that programmers have a marked tendency to assume that because two methods have similar or identical behaviour, that this implies they should share an implementation. In large code bases, its common to find methods that have identical implementations 'by chance'. By which I mean that the commonality is a side effect of requirements that could readily change. In my opinion, the best defence against this is good documentation: if a two methods have clearly documented behaviours, then even if their implementations have been fused, a subsequent programmer will have more context (and more confidence) reduplicating the code in response to further changes.

deleted 11y ago

[deleted]

agentultra 11y ago

I hate needless abstractions for the sake of removing duplication, DRY, etc. It's mad. Most of these languages don't have proper macros and so you end up going through these logical contortions to achieve a particular pattern of behavior... and it's just noise. Anyone who comes along can look at that code for hours and never know whether it performs any work (or even what that work is). A little duplication is fine until you figure out what the real problem is. The problem is usually in your data. Maybe it's not structured properly or you need to simplify the steps to transform it at an earlier stage. A good specification will go a long way to reducing the desire to introduce hapless "abstractions." An abstraction, in the mathematical sense, will hold over the domain and introducing a new one merely allows you to manipulate objects on the lower level using new algebras, predicates, etc. Code abstractions are often the leakiest abstractions. Especially the kind the author is talking about. Avoid them. Even if you have a little bit of duplication. Only start worrying about that duplication when it starts spanning compilation units/modules/whatever and is actually causing problems. Then look at the data and figure out how to structure it so you don't need that code all over the place.

Chris2048 11y ago
I think lack of things like macros is the bigger problem. There are a limited ways to abstract, or represent duplications, in a lot of languages. Sometime a comment in an issue tracker might help.

protomyth 11y ago

Isn't a bit of this governed by which programming paradigm your language uses? I get the feeling the answer and approach are much different when comparing Forth to Java.

drostie 11y ago
Yes. In theory there is a programming paradigm which fixes this, called "aspect oriented programming." I haven't seen a really accessible aspect-oriented system however.
- protomyth 11y ago
  patents will do that My point was that the way your language builds abstractions might increase or decrease the problem.
- collyw 11y ago
  Decorators in Python are pretty much the same idea. I see them used quite a lot in Flask and Django.

dandare 11y ago

Firstly, there is no such things as perfect abstractions, hence the The Law of Leaky Abstractions. Secondly, without abstraction we would be writing in machine code. So unless the authors enlightens us on what makes an abstraction wrong the whole discussion is a waste of time.

dreamdu5t 11y ago

False dichotomy. You can avoid code duplications while using the "right" abstractions.

hnbrox 11y ago
expand?

_pmf_ 11y ago

The same applies to dependencies. I've grown very reluctant towards introducing a dependency for the sake of deduplication. I'd rather have isolated modules than a de-duplicated Big Ball of Mud.

grandalf 11y ago

It's funny how often I've made this argument with even fairly experienced programmers and they seem to have a visceral reaction to the code being "less dry" than it could possibly be. Similarly, once a codebase uses several very wrong abstractions, it becomes significantly more confusing to work on, exponentially increasing cost. The temptation to use a mature library as a dependency is very strong since time is initially saved. When that library introduces the wrong abstraction, the consequences can be severe. I typically argue for (at least) creating the correct abstraction and having it wrap the mature library to constrain or properly name its behavior, and to make it less strongly coupled to the rest of the system. It doesn't just take experience to realize these sorts of things, it takes a willingness to question one's own code and imagine how it might have been done better, or how it might appear to someone who didn't write it.

TheOtherHobbes 11y ago
It's all a bit no-true-Scotsman though. abstraction != dry bad abstraction != anti-dry Abstraction is primarily about separation of concerns, not about avoiding repetition. Drying out code that's repeated all over isn't the same as creating a formal abstraction for some element of the overall logic. Which is why >once a codebase uses several very wrong abstractions, it becomes significantly more confusing to work on, exponentially increasing cost. And drying out code makes it easier to maintain, but it doesn't guarantee that the architecture isn't a mess. The problem is perhaps that CS teaches algos, and sometimes it teaches design patterns. But there's almost no useful theory of abstraction design. Design patterns are more or less as good as it gets, and all they do is give give you a cookbook of stock formulas to try. Beyond that, there's no useful way to reason about abstractions, test them for domain fit, or rate them for elegance and efficiency.
- grandalf 11y ago
  Precisely. Very well put. If anything, some developers use design patterns as a grab bag when solving a problem... a better approach is to model the solution and then be ready to "back in" to a design pattern upon noticing strong similarity or observing that the design pattern is a bit more abstract way of doing the same thing. Because of the tendency to pick a pattern first and design for the domain later, many instances of design pattern use in the wild are subtly (or not so subtly) incorrect. > Beyond that, there's no useful way to reason about abstractions, test them for domain fit, or rate them for elegance and efficiency. Very true indeed. Looking at a design through the lens of coupling and testability and modularity is a good start and can reveal many problems, but I think the real gotcha has to do with naming: Once we name a concept/abstraction we are likely to reason within that abstraction, and we rarely consider whether we are stretching it too far, or if it is even that thing anymore.

cpeterso 11y ago

I'm reminded off ESR's "curse of the gifted" email, schooling Linus Torvalds on the lack of modularization and code-sharing in Linux's driver code. :) http://lwn.net/2000/0824/a/esr-sharing.php3 http://lwn.net/2000/0824/a/esr-sharing.php3

GFK_of_xmaspast 11y ago
I'm reminded of the "I'm with those guys" comic.
hinkley 11y ago
Oh my. I recall reading this years ago. I had totally forgotten that Eric basically tells Linus to grow up and learn to use version control. Now we're all using Linus' version control.

Almaviva 11y ago

For many decisions in programming, there are tradeoffs, and relying on general rules of thumb can be much worse than judging the specific case. And heuristics are often interpreted differently by different people. Such is this discussion, imo. It's very possible to over-abstract. If something isn't extended, maybe it shouldn't be designed to be extendable, and if it isn't configured, maybe it shouldn't be configurable. On the other hand, abstractions are the heart of programming, and finding a good one is what can make you much more productive and maybe even have more fun. The feeling when I've abstracted something and got more out than I put in is maybe the best feeling there is in this craft. The difficulty is in finding the balance. I also dislike the rule of 3 as a hard general rule. There are many times when it makes sense to abstract something out of 2 uses, if there is a lot of code and clear separation and commonality involved, or you anticipate more uses later. Maybe even abstracting sub-problems out of one case makes sense, if separating the problem cleanly into two parts makes it easier to reason about. There are many times when it's a bad idea too; it depends on the specifics and it's a judgement call based on experience. (And, I would even say, there is room for personal taste in programming about this; where the sweet spot lies may vary from person to person.)

guscost 11y ago

One thing I've put a lot of thought into recently is what parts of a program should typically be abstracted, because when you come across them they might be your best bets for big wins. Top of the list: constants should probably all go in special structures with the goal of guaranteeing consistency and making them easy to understand at a glance. Everyone might know that charge code "X" means a check payment, but what happens when a new developer looks at that code with "X"s everywhere? It's more verbose to use constants.chargeCodes.CHECK_PAYMENT but nobody will misunderstand what you mean, and your IDE will be able to verify that your codes are valid. That's worth an awful lot of extra characters. Bonus win: when your legacy backend finally gets upgraded, you have the option to change to a new code for check payments, easy as pie.

hex13 11y ago

This is the old truth - premature abstraction is a root of all evil (I know original quote was "optimization", but isn't an abstraction just some "architectural optimization"? ;) On the other hand: I often see copy pasted code which have these same characteristics author wrote about: "Another additional parameter. Another new conditional. Loop until code becomes incomprehensible." If somebody writes too many conditionals, parameters and creates incomprehensible loops, then he will do it always, no matter whether in abstracted code or in copy pasted code... The problem is that most of programmers are not skilled enough to write good code. So sometimes there is a need for refactor wrong abstractions in legacy code, sometimes there is need for cleaning tons of copy-pasted code and make good abstractions... Either way - maintaining is hard.

daxfohl 11y ago

Meh, it's an easy situation to get out of though. Especially with a typed language and decent IDE: right-click, "inline function"; check your git status to see what changed, go to those files and remove unused branches (which a decent IDE should hightlight and provide 2-keystroke fixes for). 2 minutes, done. Grotesquely replicated code is a much more difficult situation to resolve. I'm guessing the OP primarily uses untyped languages. I'd offer that that is the core of the problem, not the abstractions.