5 ms·

As a programmer, if you don't create an abstraction you'll never be more than a 1x programmer. Abstraction is how to get more productive than simply how fast y

As a programmer, if you don't create an abstraction you'll never be more than a 1x programmer. Abstraction is how to get more productive than simply how fast you can type. Yes, the wrong abstraction is bad. But almost every argument whether it's for/against duplication or for/against abstraction usually starts with the hidden premise that you're stuck with whatever choice you've made and code you've written forever. The underlying issue is the fear of change and the sunk cost fallacy of already written code. If you have the wrong abstraction, you can change it. If you created too much duplication, you can remove it.

kajecounterhack 3y ago
> The underlying issue is the fear of change and the sunk cost fallacy of already written code. If you have the wrong abstraction, you can change it. It's not that trivial. Consider that the wrong abstraction is reflected into your API (common), and consider that your API has many users. You are stuck with it, or you have to convince multiple teams (or, god forbid, external customers) to migrate to a better API. This can constitute a humongous waste of SWE-hours ($$$$$) and take quarters to accomplish, assuming you can get any buy in. I think it comes down to what your organization looks like and how many users are going to be touching your code. If your abstraction is just for yourself internally and everyone else is not allowed to touch it, then fine. You will own the tech debt if the abstraction is wrong. If your abstraction has other users at your company, or external customers, it had better be the right one or at least an unavoidable stepping stone. > If you created too much duplication, you can remove it. This is actually true. Refactoring duplicated logic is a lot easier than fixing bad abstractions.
- wvenable 3y ago
  It is that trivial. There is no alternative. You either have an API or you don't and you either change it or you don't. Hand wringing over the potential of making a mistake is a waste of time and effort. You will make a mistake. You will never get it perfect. You just have to deal with it. > Refactoring duplicated logic is a lot easier than fixing bad abstractions. Then you've just created an abstraction with all that potential to be bad sometime in the future.
  - mcpeepants 3y ago
    > Then you've just created an abstraction with all that potential to be bad sometime in the future. I would argue that you've then created an abstraction, but with all the hindsight allowing you to create the _correct_ abstraction (or at least a much better chance at it approaching "correct")
  - kajecounterhack 3y ago
    > It is that trivial. > You will make a mistake. You will never get it perfect. You just have to deal with it. These two comments sound at odds. First statement says it's easy. Second statement says it's hard. We can agree that hard things don't get solved without iterating. But a productive response to abstraction (which is really API design) being hard is not to say "stop handwringing, just do it." Instead, you can employ various strategies such as preferring experienced people to do it, making sure they did a good job of gathering requirements and considered the risks of their approach, spending time testing customer/developer ergonomics, etc. You can also defer producing an abstraction until your system is a bit larger and the duplication is becoming too much to handle, since you have a larger sample size of potential uses for your abstraction to help you converge on the correct API. Good abstractions can be the difference between success and failure, between organizational velocity and technical debt quagmire. Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face.
    - wvenable 3y ago
      Trivial doesn't mean easy; it's means unimportant. And in this case I'm referring to the whole idea of abstraction or not. You're going to do it. You should do it. Sometimes you have to do it. Discussing it is pointless. Just move on to the how. > Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face. If you don't do that first "one go" how do get around to right abstraction? You're telling me you wouldn't use any intuition, planning, or thinking to build that first wrong abstraction? It seems above you just described how to do exactly that. Just writing without any abstraction in mind at all is just a huge waste of time. The only time it's painful is when you decide that you can't change anything you've done. But doing no abstraction is just as painful -- you just pay for it differently.
- Kapura 3y ago
  100% agree, especially about refactoring duplicated logic. Super-duplicated code begs to be refactored, and having many examples of the same functionality helps you build an API that is robust without adding "what-if" functionality to try to futureproof code (impossible).
- coliveira 3y ago
  Migrating to better APIs is done all the time. It is not an issue worth discussing about anymore. But even if you have to maintain an API this doesn't mean you cannot change the underlying implementation.
  - simonw 3y ago
    It absolutely is an issue worth discussing, any time you are maintaining a library with more than a few other people using it. Breaking backwards compatibility in a library with hundreds or thousands of users is not something to be taken lightly!
    - wvenable 3y ago
      I'm working with an API right now that is absolutely based on duplicated code. They have a system for querying items and the API has 3 different ways depending on the endpoint. I just found a new one the other day and I hated it -- why is this one API unnecessarily different from all the rest! I'm building a library to call this API and I've abstracted over all these differences so my callers never have to know how messed up the underlying API is -- they get a consistent experience regardless.
overgard 3y ago
I'm not sure that's true, when I look at the code bases of some of the most productive coders I can think of (John Carmack and the Doom and Quake code bases for example), they tend to be fairly conservative with how much they abstract things. There's a lot of very thoughtful data structure usage (git is another good example of this) and diligence in maintaining a coding standard (which generally has little to do with formatting), but most of the code seems to be more concrete and task focused rather than abstract. I think Casey Muratori has a good way of thinking about this, with his concept of "semantic compression" ( https://caseymuratori.com/blog_0015 https://caseymuratori.com/blog_0015 ). To me that's a lot more valuable than the ideas that you get in say something like Clean Code (which in my opinion, the popularity of that book has been a disaster for the industry)
- wvenable 3y ago
  I'm positive they don't have much in the way of duplicated code. Abstraction can be as simple as a function. It seems odd to give really good examples of abstraction and then sort of do a No true Scotsman argument on it. "It can't be abstraction because it's not terrible."
  - Kapura 3y ago
    so your argument is, "you've got to write functions, therefore abstraction is always good" ? yeah man i don't start in main() and never write another function ever again. You got me. But I also don't aggressively police duplication, accepting that I would always rather see what's happening without file-hopping. The correct abstractions will simplify code, and they will seem obvious (even if just in retrospect). Abstractions that force me to continually re-frame the problem i am trying to solve in terms of another's use case are antithetical to writing the sort of code that I do.
    - wvenable 3y ago
      You don't have to write functions. Have you ever seen a 10,000 line code that was just a single function -- I have. It also had nested if statements 5 levels deep to handle all sorts of logic with lots of duplication. It was unmaintainable. Yet it did meaningful work and it could have been refactored to a fraction of the size and do the same work. But, to be honest, when I had to fix it I just went it and fixed in the 20 places that needed changing because it was impossible to follow. You like good abstractions and you hate bad abstractions. I couldn't agree with that more.
AmpsterMan 3y ago
Wrong abstractions start off as right abstractions and slowly become wrong abstractions. What's the point in which a right abstractions becomes a wrong abstraction? Am I sure I can identify that point? Can someone else? Can someone else which has no knowledge of the original assumptions that were implied during the initial abstraction? There are two kinds of abstractions in your, the ones everyone complains about and the ones that no one has ever seen. My rule of thumb is thus: Have I repeated myself three times doing the EXACT same thing? Then CONSIDER abstracting away. Otherwise, make as many implicit dependencies explicit as possible and slightly keep repeating yourself until you are exactly repeating yourself
- wvenable 3y ago
  Your rule of thumb isn't great: If you have some important logic duplicated twice but a year from now has a bug -- are you going to remember to change it both places? But, lets be honest, you probably would not create that code in the first place -- you'd have abstracted it automatically without even thinking about it. These conversations generally tend to completely discount experience. Junior programs are often terrible at abstractions -- they either do way too much or way too little. Can I give them a hard and fast rule that they can use to never make that mistake? No, I can't. It doesn't exist. The only reason I know what's good or bad is because I've done it wrong thousands of times. That's the problem with every single one of these articles that prescribe one true solution. It's not at all that simple.
tuyiown 3y ago
I’m sorry you are totally, utterly wrong. Wrong abstractions percolates to your system, assumptions on how your abstraction is supposed to work is hard ossification that hides concrete implementation and their actual, generally simpler, contraints. Basically your refactoring work now requires to understand all user code that relies on the wrong aspects of your abstractions find a way to correct it if your lucky and make it work exactly the same way the duplicated code would have. And I didn’t mention implementations that drifts in incompatibles ways with the abstraction, a large source of errors and regrets. The good bet for productivity is recognizable implementations patterns and duplication. In the end refactoring duplicated code that had time to settle and drifted in legitimate ways to find your correct abstraction is a blast.
- wvenable 3y ago
  I disagree and I can provide an example. I'm creating a library to interface with a REST API. The creators of this REST API obviously didn't do any abstractions and they have multiple implementations for the same exact process: paged queries of items. There's no reason for them to be different -- they're just different because, I assume, different developers built them differently at different times. Nobody looked at this and said "This is all the same so we should have one single common implementation abstracting over all the endpoints." However, as the developer of the interface library, I can abstract over all the differences and give my consumers the same exact same experience regardless of the API endpoint. And that's exactly what I did. So now they're all more productive because they don't need to know all these unimportant details. They don't even need to know that there's REST API. In fact, this this REST API replaces a previous API implemented with a completely different technology and we are swapping the whole thing out with minimal changes because I abstracted it years ago. Not all abstractions are wrong. Not all concrete implementations are simpler. My goal with an abstraction is to take some else and make closer to what we need because most technology has a wide audience with wide requirements. I'm a narrow audience with narrow requirements and so I can hide the vast complexity that I simply don't care about.

kajecounterhack 3y ago

> The underlying issue is the fear of change and the sunk cost fallacy of already written code. If you have the wrong abstraction, you can change it. It's not that trivial. Consider that the wrong abstraction is reflected into your API (common), and consider that your API has many users. You are stuck with it, or you have to convince multiple teams (or, god forbid, external customers) to migrate to a better API. This can constitute a humongous waste of SWE-hours ($$$$$) and take quarters to accomplish, assuming you can get any buy in. I think it comes down to what your organization looks like and how many users are going to be touching your code. If your abstraction is just for yourself internally and everyone else is not allowed to touch it, then fine. You will own the tech debt if the abstraction is wrong. If your abstraction has other users at your company, or external customers, it had better be the right one or at least an unavoidable stepping stone. > If you created too much duplication, you can remove it. This is actually true. Refactoring duplicated logic is a lot easier than fixing bad abstractions.

wvenable 3y ago
It is that trivial. There is no alternative. You either have an API or you don't and you either change it or you don't. Hand wringing over the potential of making a mistake is a waste of time and effort. You will make a mistake. You will never get it perfect. You just have to deal with it. > Refactoring duplicated logic is a lot easier than fixing bad abstractions. Then you've just created an abstraction with all that potential to be bad sometime in the future.
- mcpeepants 3y ago
  > Then you've just created an abstraction with all that potential to be bad sometime in the future. I would argue that you've then created an abstraction, but with all the hindsight allowing you to create the _correct_ abstraction (or at least a much better chance at it approaching "correct")
- kajecounterhack 3y ago
  > It is that trivial. > You will make a mistake. You will never get it perfect. You just have to deal with it. These two comments sound at odds. First statement says it's easy. Second statement says it's hard. We can agree that hard things don't get solved without iterating. But a productive response to abstraction (which is really API design) being hard is not to say "stop handwringing, just do it." Instead, you can employ various strategies such as preferring experienced people to do it, making sure they did a good job of gathering requirements and considered the risks of their approach, spending time testing customer/developer ergonomics, etc. You can also defer producing an abstraction until your system is a bit larger and the duplication is becoming too much to handle, since you have a larger sample size of potential uses for your abstraction to help you converge on the correct API. Good abstractions can be the difference between success and failure, between organizational velocity and technical debt quagmire. Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face.
  - wvenable 3y ago
    Trivial doesn't mean easy; it's means unimportant. And in this case I'm referring to the whole idea of abstraction or not. You're going to do it. You should do it. Sometimes you have to do it. Discussing it is pointless. Just move on to the how. > Saying "we should always build abstractions" when it's difficult to build them correctly in one go sounds totally wrong on its face. If you don't do that first "one go" how do get around to right abstraction? You're telling me you wouldn't use any intuition, planning, or thinking to build that first wrong abstraction? It seems above you just described how to do exactly that. Just writing without any abstraction in mind at all is just a huge waste of time. The only time it's painful is when you decide that you can't change anything you've done. But doing no abstraction is just as painful -- you just pay for it differently.
Kapura 3y ago
100% agree, especially about refactoring duplicated logic. Super-duplicated code begs to be refactored, and having many examples of the same functionality helps you build an API that is robust without adding "what-if" functionality to try to futureproof code (impossible).
coliveira 3y ago
Migrating to better APIs is done all the time. It is not an issue worth discussing about anymore. But even if you have to maintain an API this doesn't mean you cannot change the underlying implementation.
- simonw 3y ago
  It absolutely is an issue worth discussing, any time you are maintaining a library with more than a few other people using it. Breaking backwards compatibility in a library with hundreds or thousands of users is not something to be taken lightly!
  - wvenable 3y ago
    I'm working with an API right now that is absolutely based on duplicated code. They have a system for querying items and the API has 3 different ways depending on the endpoint. I just found a new one the other day and I hated it -- why is this one API unnecessarily different from all the rest! I'm building a library to call this API and I've abstracted over all these differences so my callers never have to know how messed up the underlying API is -- they get a consistent experience regardless.

overgard 3y ago

I'm not sure that's true, when I look at the code bases of some of the most productive coders I can think of (John Carmack and the Doom and Quake code bases for example), they tend to be fairly conservative with how much they abstract things. There's a lot of very thoughtful data structure usage (git is another good example of this) and diligence in maintaining a coding standard (which generally has little to do with formatting), but most of the code seems to be more concrete and task focused rather than abstract. I think Casey Muratori has a good way of thinking about this, with his concept of "semantic compression" ( https://caseymuratori.com/blog_0015 https://caseymuratori.com/blog_0015 ). To me that's a lot more valuable than the ideas that you get in say something like Clean Code (which in my opinion, the popularity of that book has been a disaster for the industry)

wvenable 3y ago
I'm positive they don't have much in the way of duplicated code. Abstraction can be as simple as a function. It seems odd to give really good examples of abstraction and then sort of do a No true Scotsman argument on it. "It can't be abstraction because it's not terrible."
- Kapura 3y ago
  so your argument is, "you've got to write functions, therefore abstraction is always good" ? yeah man i don't start in main() and never write another function ever again. You got me. But I also don't aggressively police duplication, accepting that I would always rather see what's happening without file-hopping. The correct abstractions will simplify code, and they will seem obvious (even if just in retrospect). Abstractions that force me to continually re-frame the problem i am trying to solve in terms of another's use case are antithetical to writing the sort of code that I do.
  - wvenable 3y ago
    You don't have to write functions. Have you ever seen a 10,000 line code that was just a single function -- I have. It also had nested if statements 5 levels deep to handle all sorts of logic with lots of duplication. It was unmaintainable. Yet it did meaningful work and it could have been refactored to a fraction of the size and do the same work. But, to be honest, when I had to fix it I just went it and fixed in the 20 places that needed changing because it was impossible to follow. You like good abstractions and you hate bad abstractions. I couldn't agree with that more.

AmpsterMan 3y ago

Wrong abstractions start off as right abstractions and slowly become wrong abstractions. What's the point in which a right abstractions becomes a wrong abstraction? Am I sure I can identify that point? Can someone else? Can someone else which has no knowledge of the original assumptions that were implied during the initial abstraction? There are two kinds of abstractions in your, the ones everyone complains about and the ones that no one has ever seen. My rule of thumb is thus: Have I repeated myself three times doing the EXACT same thing? Then CONSIDER abstracting away. Otherwise, make as many implicit dependencies explicit as possible and slightly keep repeating yourself until you are exactly repeating yourself

wvenable 3y ago
Your rule of thumb isn't great: If you have some important logic duplicated twice but a year from now has a bug -- are you going to remember to change it both places? But, lets be honest, you probably would not create that code in the first place -- you'd have abstracted it automatically without even thinking about it. These conversations generally tend to completely discount experience. Junior programs are often terrible at abstractions -- they either do way too much or way too little. Can I give them a hard and fast rule that they can use to never make that mistake? No, I can't. It doesn't exist. The only reason I know what's good or bad is because I've done it wrong thousands of times. That's the problem with every single one of these articles that prescribe one true solution. It's not at all that simple.

tuyiown 3y ago

I’m sorry you are totally, utterly wrong. Wrong abstractions percolates to your system, assumptions on how your abstraction is supposed to work is hard ossification that hides concrete implementation and their actual, generally simpler, contraints. Basically your refactoring work now requires to understand all user code that relies on the wrong aspects of your abstractions find a way to correct it if your lucky and make it work exactly the same way the duplicated code would have. And I didn’t mention implementations that drifts in incompatibles ways with the abstraction, a large source of errors and regrets. The good bet for productivity is recognizable implementations patterns and duplication. In the end refactoring duplicated code that had time to settle and drifted in legitimate ways to find your correct abstraction is a blast.

wvenable 3y ago
I disagree and I can provide an example. I'm creating a library to interface with a REST API. The creators of this REST API obviously didn't do any abstractions and they have multiple implementations for the same exact process: paged queries of items. There's no reason for them to be different -- they're just different because, I assume, different developers built them differently at different times. Nobody looked at this and said "This is all the same so we should have one single common implementation abstracting over all the endpoints." However, as the developer of the interface library, I can abstract over all the differences and give my consumers the same exact same experience regardless of the API endpoint. And that's exactly what I did. So now they're all more productive because they don't need to know all these unimportant details. They don't even need to know that there's REST API. In fact, this this REST API replaces a previous API implemented with a completely different technology and we are swapping the whole thing out with minimal changes because I abstracted it years ago. Not all abstractions are wrong. Not all concrete implementations are simpler. My goal with an abstraction is to take some else and make closer to what we need because most technology has a wide audience with wide requirements. I'm a narrow audience with narrow requirements and so I can hide the vast complexity that I simply don't care about.