4 ms·

I'm surprised I'm the first to bring this up, but the "over-abstract" example feels more like "over-specified". The actual operation being defined works on the

I'm surprised I'm the first to bring this up, but the "over-abstract" example feels more like "over-specified". The actual operation being defined works on the same level of abstraction, since it defines precisely the same operation. The first has just specified extra tweakable aspects of its execution via the argument list. I'm not saying it's not bad, I just don't think it's because it is "too abstract" compared to the simpler solution. Abstraction in my mind is fundamentally about the "higher-orderness" of a thing. These two average methods are just as abstract as the other, since one is not a higher-order operation than the other. I would use the word over-abstract if one was to write a program modeling a dog-walking business (a very specific thing), by writing a system which models actions on entities (a very abstract thing), where walking is an action which may involve one or more entities, and dogs, humans, employees, and customers are all entities. If the core thing you want to do is just a single concretion of the system that you actually built, then you "over-abstracted". I feel like we should not discourage the practice of abstraction, since that's our business. I literally get paid to think about the real world in terms of abstraction and write it up into a computer. Young engineers should not be taught to fear "over-abstraction".

MaxBarraclough 6y ago
> The actual operation being defined works on the same level of abstraction, since it defines precisely the same operation. Agreed. This confusion is something Zed Shaw wrote about in blog post called Indirection Is Not Abstraction [0]. Surprisingly it seems it was never discussed properly on HN [1], but it was discussed elsewhere [2][3]. (There's also another blog post by this name by another blogger, by independent reinvention/coincidence [4].) > Young engineers should not be taught to fear "over-abstraction". Disagree. As you just showed, unnecessary abstraction is bad. Ineffective abstractions are also bad. It's not easy to get right. [0] https://web.archive.org/web/20160304022133/http://zedshaw.com/archive/indirection-is-not-abstraction/ https://web.archive.org/web/20160304022133/http://zedshaw.co... [1] https://news.ycombinator.com/from?site=zedshaw.com&next=8820179 https://news.ycombinator.com/from?site=zedshaw.com&next=8820... [2] https://www.reddit.com/r/programming/comments/38hobm/zed_shaw_indirection_is_not_abstraction/ https://www.reddit.com/r/programming/comments/38hobm/zed_sha... [3] https://lobste.rs/s/ja5ihv/indirection_is_not_abstraction https://lobste.rs/s/ja5ihv/indirection_is_not_abstraction [4] https://news.ycombinator.com/item?id=18344033 https://news.ycombinator.com/item?id=18344033
- gen220 6y ago
  Thank you for these excellent links. This debate often takes the concrete form of “one large imperative function with 100 lines” vs “5 functions with 20 lines”. Many people who separate out logic for the sake of minimizing lines-per-function unfortunately do so by introducing indirection, rather than abstractions, and thereby make the program more challenging to reason about and test. But, the reason the debate is never-ending is because the question isn’t sufficiently defined! :) I like to think of a program like a tree (main is root, each function is a node, calls are edges). Each sub tree should be a bounded context, in that (ideally) you only have to think about parameters defined in that sub-tree. Leaves implement the “nitty-gritty” (mainly I/O, number-crunching, and complex transformations), and are heavily tested. Each node that isn’t a leaf is either an abstraction composing leaves, or an abstraction composing abstractions. Unit tests for leaves test the nitty gritty, unit tests for non-leaves must only test composition. I find that human-readable modules have some limits (number of children, height of the tree). You can violate those limits sometimes, but only if you provide some assistance in the form of comments. Sometimes, a 100-line function is not composing many distinct nitty-gritty ideas. It just really takes 100 lines to express “write this model to the database”.
  - MaxBarraclough 6y ago
    I agree that relatively large functions aren't always an evil. As you say, sometimes there isn't a tidy way to further decompose it. At the same time though I don't think it's always a sin to write a function just for decomposition, without it doing any abstraction. A 'helper function' might be tightly bound to some other function, i.e. the helper function is sensitive to the internal workings of the function it serves, and is not intended to be called from anywhere else. And there's still no excuse for source files that are 5000 lines long, of course. HN discussion of John Carmack's thoughts on how long functions are sometimes preferable: https://news.ycombinator.com/item?id=8374345 https://news.ycombinator.com/item?id=8374345
rpastuszak 6y ago
> I feel like we should not discourage the practice of abstraction, since that's our business. I literally get paid to think about the real world in terms of abstraction and write it up into a computer. Young engineers should not be taught to fear "over-abstraction". I agree with most of your comment, but in my experience (and, I think many may echo the same sentiment) over-abstracting has been always more dangerous than the opposite. I’d choose a messy, duplicated piece of code over an overly abstracted 10x developer made mess, every single time. > I literally get paid to think about the real world in terms of abstraction and write it up into a computer. I love software engineering, but the most satisfying moments in my work involve removing code or not relying on tech to solve problems whatsoever.
searchableguy 6y ago
I think with modern editors, the cost of code duplication is lower than the cost of a tight over abstracted code linked everywhere in your system. You can easily find and replace instances of duplicated code as it would be isolated and can be automated to some degree.
jesseduffield 6y ago
I consider abstraction to be about giving various concrete things the same representation. In this case I'm saying that we're over-abstracting by pulling too much of the dissimilar code between the examples into the one representation (i.e. the method). I would also say with the `average` methods it's not quite the same operation, despite having the same name. The over-abstracted method had quite a bit more going on internally than the minimal abstraction, and had a different interface. With your dog walking example, I'd say each of your listed abstractions would be 'the right abstraction' because you're not bundling up dissimilar things into a single representation as if they were similar. Specifically with dogs/humans, my example about circles/squares is the same: both might conform to the same interface, but you wouldn't want to represent them with the same class. I agree that doing so is perhaps a special kind of mistake for which there may be a better term than 'over-abstraction', though it's not obvious to me what that term would be (over-specified doesn't quite sound right to me).
- codemonkey-zeta 6y ago
  In this case I'm saying that we're over-abstracting by pulling too much of the dissimilar code between the examples into the one representation (i.e. the method). Ok it sounds like we agree on what is wrong with the example. "Over-abstract" still feels like the wrong word for that, because the actual problem is that we have ruined our abstraction layer with junk about lower layers. Average is an operation on lists of numbers, but ignore_nulls is a feature of the members of the list in your programming language, same with the Type argument. The members of data structures intuitively (maybe not always) exist at a lower level of abstraction. I would be more inclined to call this something similar to "partial-abstraction", because the programmer didn't take the time to remove all the semantics of the lower levels from the interface to the higher level.
  - jesseduffield 6y ago
    I just came across the term of https://en.wikipedia.org/wiki/Leaky_abstraction https://en.wikipedia.org/wiki/Leaky_abstraction. I think that term captures what we're talking about, do you agree?

MaxBarraclough 6y ago

> The actual operation being defined works on the same level of abstraction, since it defines precisely the same operation. Agreed. This confusion is something Zed Shaw wrote about in blog post called Indirection Is Not Abstraction [0]. Surprisingly it seems it was never discussed properly on HN [1], but it was discussed elsewhere [2][3]. (There's also another blog post by this name by another blogger, by independent reinvention/coincidence [4].) > Young engineers should not be taught to fear "over-abstraction". Disagree. As you just showed, unnecessary abstraction is bad. Ineffective abstractions are also bad. It's not easy to get right. [0] https://web.archive.org/web/20160304022133/http://zedshaw.com/archive/indirection-is-not-abstraction/ https://web.archive.org/web/20160304022133/http://zedshaw.co... [1] https://news.ycombinator.com/from?site=zedshaw.com&next=8820179 https://news.ycombinator.com/from?site=zedshaw.com&next=8820... [2] https://www.reddit.com/r/programming/comments/38hobm/zed_shaw_indirection_is_not_abstraction/ https://www.reddit.com/r/programming/comments/38hobm/zed_sha... [3] https://lobste.rs/s/ja5ihv/indirection_is_not_abstraction https://lobste.rs/s/ja5ihv/indirection_is_not_abstraction [4] https://news.ycombinator.com/item?id=18344033 https://news.ycombinator.com/item?id=18344033

gen220 6y ago
Thank you for these excellent links. This debate often takes the concrete form of “one large imperative function with 100 lines” vs “5 functions with 20 lines”. Many people who separate out logic for the sake of minimizing lines-per-function unfortunately do so by introducing indirection, rather than abstractions, and thereby make the program more challenging to reason about and test. But, the reason the debate is never-ending is because the question isn’t sufficiently defined! :) I like to think of a program like a tree (main is root, each function is a node, calls are edges). Each sub tree should be a bounded context, in that (ideally) you only have to think about parameters defined in that sub-tree. Leaves implement the “nitty-gritty” (mainly I/O, number-crunching, and complex transformations), and are heavily tested. Each node that isn’t a leaf is either an abstraction composing leaves, or an abstraction composing abstractions. Unit tests for leaves test the nitty gritty, unit tests for non-leaves must only test composition. I find that human-readable modules have some limits (number of children, height of the tree). You can violate those limits sometimes, but only if you provide some assistance in the form of comments. Sometimes, a 100-line function is not composing many distinct nitty-gritty ideas. It just really takes 100 lines to express “write this model to the database”.
- MaxBarraclough 6y ago
  I agree that relatively large functions aren't always an evil. As you say, sometimes there isn't a tidy way to further decompose it. At the same time though I don't think it's always a sin to write a function just for decomposition, without it doing any abstraction. A 'helper function' might be tightly bound to some other function, i.e. the helper function is sensitive to the internal workings of the function it serves, and is not intended to be called from anywhere else. And there's still no excuse for source files that are 5000 lines long, of course. HN discussion of John Carmack's thoughts on how long functions are sometimes preferable: https://news.ycombinator.com/item?id=8374345 https://news.ycombinator.com/item?id=8374345

rpastuszak 6y ago

> I feel like we should not discourage the practice of abstraction, since that's our business. I literally get paid to think about the real world in terms of abstraction and write it up into a computer. Young engineers should not be taught to fear "over-abstraction". I agree with most of your comment, but in my experience (and, I think many may echo the same sentiment) over-abstracting has been always more dangerous than the opposite. I’d choose a messy, duplicated piece of code over an overly abstracted 10x developer made mess, every single time. > I literally get paid to think about the real world in terms of abstraction and write it up into a computer. I love software engineering, but the most satisfying moments in my work involve removing code or not relying on tech to solve problems whatsoever.

searchableguy 6y ago

I think with modern editors, the cost of code duplication is lower than the cost of a tight over abstracted code linked everywhere in your system. You can easily find and replace instances of duplicated code as it would be isolated and can be automated to some degree.

jesseduffield 6y ago

I consider abstraction to be about giving various concrete things the same representation. In this case I'm saying that we're over-abstracting by pulling too much of the dissimilar code between the examples into the one representation (i.e. the method). I would also say with the `average` methods it's not quite the same operation, despite having the same name. The over-abstracted method had quite a bit more going on internally than the minimal abstraction, and had a different interface. With your dog walking example, I'd say each of your listed abstractions would be 'the right abstraction' because you're not bundling up dissimilar things into a single representation as if they were similar. Specifically with dogs/humans, my example about circles/squares is the same: both might conform to the same interface, but you wouldn't want to represent them with the same class. I agree that doing so is perhaps a special kind of mistake for which there may be a better term than 'over-abstraction', though it's not obvious to me what that term would be (over-specified doesn't quite sound right to me).

codemonkey-zeta 6y ago
In this case I'm saying that we're over-abstracting by pulling too much of the dissimilar code between the examples into the one representation (i.e. the method). Ok it sounds like we agree on what is wrong with the example. "Over-abstract" still feels like the wrong word for that, because the actual problem is that we have ruined our abstraction layer with junk about lower layers. Average is an operation on lists of numbers, but ignore_nulls is a feature of the members of the list in your programming language, same with the Type argument. The members of data structures intuitively (maybe not always) exist at a lower level of abstraction. I would be more inclined to call this something similar to "partial-abstraction", because the programmer didn't take the time to remove all the semantics of the lower levels from the interface to the higher level.
- jesseduffield 6y ago
  I just came across the term of https://en.wikipedia.org/wiki/Leaky_abstraction https://en.wikipedia.org/wiki/Leaky_abstraction. I think that term captures what we're talking about, do you agree?