10 ms·

I see this fact used pretty often to dismiss the idea that we should use abstractions at all, and I think that's pretty wrongheaded. As an example, it's pretty

I see this fact used pretty often to dismiss the idea that we should use abstractions at all, and I think that's pretty wrongheaded. As an example, it's pretty common to hear that people should prefer writing raw SQL to using an ORM or query builder because those are both "leaky abstractions" Abstractions aren't necessarily only there so that you don't need to understand anything about what's being abstracted. They are there so that your code can get away from nitty-gritty implementation details, and be more focused on the problem domain. What would you rather see when you're coming into a new codebase: const formData = new FormData(); formData.append('name', 'Widget'); fetch(`/api/widgets/${id}`, { method: 'POST', body: formData }) or: Widget.save({ name: 'Widget' });

mannykannot 7y ago
SQL is itself a high-level abstraction, and I have seen cases where an ill-conceived data abstraction layer has just added complexity without improving the quality of abstraction (I have also seen many cases where poorly-designed database schema have nullified the relational model's power of abstraction.) With regard to your question, I want to be able to see both the abstraction and the implementation, as, depending on what I am doing at any given time, I might need to understand either.
sergiosgc 7y ago
> As an example, it's pretty common to hear that people should prefer writing raw SQL to using an ORM or query builder because those are both "leaky abstractions" The takeaway of ORMs being leaky abstractions is that they should expose the lower abstraction. An ORM builds the query for you 90% of the time, but when (not if) you need to write raw SQL, the ORM should be able to integrate the raw SQL output in its flow. An example of an ORM that does this beautifully is SQLAlchemy [https://www.sqlalchemy.org/ https://www.sqlalchemy.org/]
gchamonlive 7y ago
Abstractions are also great to create sections of testable code separate from other parts harder to test (database comm, requests, IO...) But they can also get in the way when they are too coupled with the code or when the layers of abstractions get out of hand. "the only problem that more abstraction can't solve is too much abstraction" On a side note on orm leaky abstractions, that was particularly clear to me when using cakephp. It was really hard to debug what was going on in the background and getting the actual query being run with its placeholders in place is a nightmare. SQLalchemy in contrasts is extremely close to the actual query and with code you can find online you can actually generate the sql string and it is always very close to what I had in mind in the first place, thus properly integrating the extra expressivity python has into building sql queries.
XCabbage 7y ago
Some abstractions are basically perfect and don't require you to understand the layer beneath them at all. Some have that as their ideal, but are imperfect, and sometimes require you to understand a few details of the layer below. But others - the best example that I know of is SCSS - aren't even trying to eliminate the need to understand the layer below. Everything you need to know to write CSS, you still need to know to write SCSS, and then SCSS adds more on top. It's a convenient tool for power users and I wouldn't want to be without it, but for a beginner, using SCSS instead of CSS means you have strictly more to learn; I'd suggest that a newbie on my team not touch SCSS until they've done a few days' work with raw CSS. While less clear-cut than SCSS/CSS, it seems to me that ORMs/SQL have a similar relationship. Frankly, without knowing SQL, you can't hope to competently use an ORM; you won't know what sort of queries and updates it's capable of or what its performance characteristics will be, let alone more subtle stuff like what indexes to create or how the database's transaction model works. But even once you DO understand the SQL layer, getting to grips with how ORMs work is a major additional hurdle. For that reason, while I am comfortable using an ORM for projects I work on, I wouldn't recommend that a beginner do the same. If I did, I'd be doubling the amount they have to learn, and creating a risk that they'll give up in despair trying to figure out how something poorly-documented works at the ORM layer because they don't have the knowledge of the underlying SQL layer to intuitively guess what their ORM code must be doing under the hood. It's perhaps more radical than anything I believe, but I think you can reasonably go further and make the case that the extra learning curve imposed by ORMs is not worth the minor convenience benefit they offer to the proficient, and that for that reason, it's generally better not to use them. The fact that they leak the entire layer underneath them is part of that argument, but not all of it; the other part is that the layer they build on top of that is difficult-to-learn and only adds a small bit of convenience in exchange.
- BigJono 7y ago
  Some abstractions don't seem to give anything useful at all other than homogeneity across code bases and slightly nicer looking (but not simpler) code, and yet they still see wide use.
  - ryanbrunner 7y ago
marcosdumay 7y ago
The main problem with ORM is not that they are a leaky abstraction. It's that for any minimally complex problem they are a bad abstraction. Abstractions usually empower their user, ORMs remove power.
- ryanbrunner 7y ago
  Good abstractions allow themselves to get out of the way when necessary. There are for sure bad examples of abstractions, but there's also plenty of ORMs that let you bypass the abstraction completely when needed (or use the parts of it still applicable). I also don't really understand your last statement. Abstractions, pretty much by definition, never allow you to do more than the underlying technology they're abstracting. They necessarily limit power in order to make code more comprehensible and maintainable. That's true for any abstraction I can think of, leaky or not.
  - learc83 7y ago
    In addition to getting out od the way I prefer when an ORM allows me to extend it to build my own abstractions. For example in Rails when I have a complicated query, I can make write some SQL, stick it in a view, and make a view backed model. Then I can keep using Active Record to make it easier use the query in multiple places, easily compose additional queries etc...
- captainbland 7y ago
  To be fair, most database accesses many people are going to be doing aren't even minimally complex and having an ORM saves you a lot of tedious, boring, potentially error-prone work in those cases. This shouldn't stop developers from actually writing a SQL query if the task at hand warrants it and using an ORM doesn't usually stop you from doing that.
- kyberias 7y ago
  > Abstractions usually empower their user, ORMs remove power. That is an opinion. One could also consider that the fact that you can use programming language to model your database provides a lot of expressive power.
ZeroFries 7y ago
You can't really dismiss abstractions anyways, because everything is an abstraction. SQL is most definitely an abstraction, even machine code and assembly are abstractions. Abstractions aren't reality (the map is not the territory and all that), but they capture some aspect of reality, in some sense, and if that aspect happens to be the part you care about, they're useful. It's not about not using abstractions, it's about choosing the correct abstraction for the given problem domain.
- Frost1x 7y ago
  Abstractions should also simplify. If an abstraction makes something unnecessarily complex, then using abstraction for the sake of abstraction isn't useful. In the spirit of YAGNI, you shouldn't try planning too far ahead with your abstractions. From my experience, sometimes your clever planning all falls into place, but most the time something significant shifts in client demands and your abstractions don't make much sense anymore and add nothing but a layer of complexity to deal with that didn't need to be there to begin with.
icxa 7y ago
The argument against ORMs aren't that they are a leaky abstraction (although they most definitely are). The argument is although they appear to offer you value up front, they cost you much more down the road. The moment you have a hot path query that needs optimizing, you are dropping down to your ORM's "raw sql" mode. Then you do it again. Then again. Then you are ripping out the ORM and spending cycles refactoring it out of your code and replacing it with simpler abstractions. I always find the people who don't believe writing raw SQL is preferable to using an ORM is usually down to a lack of experience or because they have been coerced to use ORMs by their "enterprise grade" language (usually your C# developers of the world, no offense to you all but if every time you look up examples of data operations it's dealing with EntityFramework you are probably going to wind up with a lot of devs using EntityFramework). The ORMers, as I call them, don't have confidence in their own SQL ability, and they haven't experienced the aforementioned situation enough times to realize you are better off starting with SQL to begin with. The realization is, if I could summarize it: yes all abstractions are at least somewhat leaky, so you are better to use simpler ones than complex ones (and if you need something more complex, compose it from simpler ones)
- jtdev 7y ago
  This is reflective of my experience with the ORM zealots as well... SQL is already an abstraction, and it happens to be one of the most well thought out, battle tested abstractions we have in the software world... it drives me nuts when junior devs and devs with poor SQL abilities want to pollute a codebase with EF, Hibernate, etc. without understanding if it’s necessary or even helpful beyond simply allowing the developer to continue onward without spending time becoming proficient in SQL. That said, there are scenarios where an ORM is helpful, but I personally find those occasions to be the exception.
  - ajross 7y ago
    > SQL is already an abstraction Still a leaky one, though. Sure, the syntax likes to pretend that you "just describe what you want", but in practice getting queries on large data sets to perform well involves layers of optimization, all the way down to deciding on how data is partitioned between storage backends and what the on-disk format of it and its indexes needs to be. In fact I view SQL as a somewhat thin abstraction, with a bunch of unavoidable (or at least unavoided) holes. Maybe that's part of the reason it's been so successful.
Nasrudith 7y ago
Really I think the point is to know what you are abstracting for and not simply dogmatism abstraction for the sake of abstraction or "purity". If you expect to have a system which handles the low level details that is fine. If you don't want it to possible to mess up state consistency at the cost of possible performance fine. However, if you expect abstraction to mean latency doesn't vary at all regardless of if it is on chip cache, RAM, hard drive you are going to have a bad time unless you are dealing with a library that specifically tries to mask timing attacks.
jrochkind1 7y ago
Indeed software engineering pretty much consists of abstractions. (And computer science is basically the science of formalized abstraction, that's basically what CS is). If we didn't have any abstractions at all.. we'd all be writing machine code in 1s and 0s. (and even that is an abstraction, of course...) That wouldn't get us very far. People, including in this thread, also often talk about a particular abstraction being "a leaky abstraction". Implying some abstractions are, some aren't. Whereas in fact the 'law' quoted says that all "non-trivial abstractions, to some degree, are leaky." They're all leaky, the map is never the territory. But it's the business we're in. (But some abstractions are more useful, more successful, and/or more dangerous than others... in a given context of usage, always in a given context).
- alexandercrohde 7y ago
  I agree all engineering is abstractions. Every function, data-structure, program. >> [People] talk about a particular abstraction being "a leaky abstraction". Implying some abstractions are, some aren't. There are degrees of leakiness though, which matter a great deal. The concept of a "file" is an abstraction, instead we could all write direct to disk sectors. But its value outweighs the leak associated with it. Other abstractions aren't always so useful, sometimes a coworker makes a class that's riddled with exceptions and caveats.
  - jrochkind1 7y ago
    Some abstractions are definitely more useful/successful/robust than others. (I think I want to say usually for a particular context, not as a universal/existential thing. Sometimes that context grows to be everyone's... we all use a 'file' abstraction because it was so useful the entire field adopted it, and stopped considering alternative abstractions, and everyone takes as a starting assumption to work within the context where 'file' is useful (except in extreme edge cases maybe) -- for better or worse.) Deciding which is which, whether a particular abstraction's value outweighs the "leak" in particular contexts or in "general" -- well, that's what endless internet debate is made of. It's usually not clear as a consensus until quite some time has passed (and even then heretics can challenge it). I agree it's the right question, but it can't be answered simply by saying "leaky abstraction".
cletus 7y ago
They (and maybe you?) are missing the point of the Law of Leaky Abstractions here. The point is that no matter how good your abstraction there will be corner cases where it gets in the way. This is also a great example of the perfect being an enemy of the good. Some people will try to get that last gap out of their abstraction to the point of it then becoming a little bit worse in all cases and still not being perfect. The point is that your abstraction shouldn't get in your way. It should be easy to bypass when you need to. The problem with ORMs in particular is that this tends to be impossible. Any ORM I've used has added a caching layer so then you can't run raw SQL queries and the ORM is never as expressive enough as raw SQL. Sometimes you just want a subset of columns but ORMs don't tend to support that since a table is mapped to an object and it's either all or nothing. Years ago, this is why I eschewed the likes of Hibernate and preferred things like iBatis.
- tomnipotent 7y ago
  > Sometimes you just want a subset of columns There isn't a mature ORM I know of that doesn't support this, as either partial bindings to the mapped object (with empty properties) or whatever k/v primitive is supported by the language.
barbecue_sauce 7y ago
I think `fetch` is the real leaky abstraction.
Bodell 7y ago
To agree on this inside of a broader context: I would argue that there are so many abstractions in language (“good”, “evil”, “god”, even less lofty things such as “color”, “heat”, “weather”), science ("gravity" would probably an arguable but interesting example as well as one that came up recently with a neighbor of mine), and even basic functional things (including “plumbing”, “electricity”, and even “computers” at quite literally every level of understanding above mathematics), so in keeping all of that in mind possibly our entire sense of being and living in the world would be completely debilitating without making use of an innumerable amount of abstractions. That said there are always valid arguments against the use of particular abstractions, as they can be seen in practice as being largely more detrimental that helpful. Many make this argument against "god". My High school physics teacher made the argument against the use of “centripetal force.” I myself like to remind people that they probably know far less about the world around them than they think simply because that have replaced conceptual knowledge with a definition and categorical knowledge. Though at the end of the day, I feel that abstractions in general are quite harmless, as long as you are aware of the fact that they are just that “abstractions.”
hyeomans 7y ago
I would rather see the first one. I'm no expert on software development, however I've seen that as time progresses new requirements are added. Cross-cutting requirements are specially hard to implement on your second option, imagine a request id has to be sent. My 2c anyway

mannykannot 7y ago

SQL is itself a high-level abstraction, and I have seen cases where an ill-conceived data abstraction layer has just added complexity without improving the quality of abstraction (I have also seen many cases where poorly-designed database schema have nullified the relational model's power of abstraction.) With regard to your question, I want to be able to see both the abstraction and the implementation, as, depending on what I am doing at any given time, I might need to understand either.

sergiosgc 7y ago

> As an example, it's pretty common to hear that people should prefer writing raw SQL to using an ORM or query builder because those are both "leaky abstractions" The takeaway of ORMs being leaky abstractions is that they should expose the lower abstraction. An ORM builds the query for you 90% of the time, but when (not if) you need to write raw SQL, the ORM should be able to integrate the raw SQL output in its flow. An example of an ORM that does this beautifully is SQLAlchemy [https://www.sqlalchemy.org/ https://www.sqlalchemy.org/]

gchamonlive 7y ago

Abstractions are also great to create sections of testable code separate from other parts harder to test (database comm, requests, IO...) But they can also get in the way when they are too coupled with the code or when the layers of abstractions get out of hand. "the only problem that more abstraction can't solve is too much abstraction" On a side note on orm leaky abstractions, that was particularly clear to me when using cakephp. It was really hard to debug what was going on in the background and getting the actual query being run with its placeholders in place is a nightmare. SQLalchemy in contrasts is extremely close to the actual query and with code you can find online you can actually generate the sql string and it is always very close to what I had in mind in the first place, thus properly integrating the extra expressivity python has into building sql queries.

XCabbage 7y ago

Some abstractions are basically perfect and don't require you to understand the layer beneath them at all. Some have that as their ideal, but are imperfect, and sometimes require you to understand a few details of the layer below. But others - the best example that I know of is SCSS - aren't even trying to eliminate the need to understand the layer below. Everything you need to know to write CSS, you still need to know to write SCSS, and then SCSS adds more on top. It's a convenient tool for power users and I wouldn't want to be without it, but for a beginner, using SCSS instead of CSS means you have strictly more to learn; I'd suggest that a newbie on my team not touch SCSS until they've done a few days' work with raw CSS. While less clear-cut than SCSS/CSS, it seems to me that ORMs/SQL have a similar relationship. Frankly, without knowing SQL, you can't hope to competently use an ORM; you won't know what sort of queries and updates it's capable of or what its performance characteristics will be, let alone more subtle stuff like what indexes to create or how the database's transaction model works. But even once you DO understand the SQL layer, getting to grips with how ORMs work is a major additional hurdle. For that reason, while I am comfortable using an ORM for projects I work on, I wouldn't recommend that a beginner do the same. If I did, I'd be doubling the amount they have to learn, and creating a risk that they'll give up in despair trying to figure out how something poorly-documented works at the ORM layer because they don't have the knowledge of the underlying SQL layer to intuitively guess what their ORM code must be doing under the hood. It's perhaps more radical than anything I believe, but I think you can reasonably go further and make the case that the extra learning curve imposed by ORMs is not worth the minor convenience benefit they offer to the proficient, and that for that reason, it's generally better not to use them. The fact that they leak the entire layer underneath them is part of that argument, but not all of it; the other part is that the layer they build on top of that is difficult-to-learn and only adds a small bit of convenience in exchange.

BigJono 7y ago
Some abstractions don't seem to give anything useful at all other than homogeneity across code bases and slightly nicer looking (but not simpler) code, and yet they still see wide use.
- ryanbrunner 7y ago

marcosdumay 7y ago

The main problem with ORM is not that they are a leaky abstraction. It's that for any minimally complex problem they are a bad abstraction. Abstractions usually empower their user, ORMs remove power.

ryanbrunner 7y ago
Good abstractions allow themselves to get out of the way when necessary. There are for sure bad examples of abstractions, but there's also plenty of ORMs that let you bypass the abstraction completely when needed (or use the parts of it still applicable). I also don't really understand your last statement. Abstractions, pretty much by definition, never allow you to do more than the underlying technology they're abstracting. They necessarily limit power in order to make code more comprehensible and maintainable. That's true for any abstraction I can think of, leaky or not.
- learc83 7y ago
  In addition to getting out od the way I prefer when an ORM allows me to extend it to build my own abstractions. For example in Rails when I have a complicated query, I can make write some SQL, stick it in a view, and make a view backed model. Then I can keep using Active Record to make it easier use the query in multiple places, easily compose additional queries etc...
captainbland 7y ago
To be fair, most database accesses many people are going to be doing aren't even minimally complex and having an ORM saves you a lot of tedious, boring, potentially error-prone work in those cases. This shouldn't stop developers from actually writing a SQL query if the task at hand warrants it and using an ORM doesn't usually stop you from doing that.
kyberias 7y ago
> Abstractions usually empower their user, ORMs remove power. That is an opinion. One could also consider that the fact that you can use programming language to model your database provides a lot of expressive power.

ZeroFries 7y ago

You can't really dismiss abstractions anyways, because everything is an abstraction. SQL is most definitely an abstraction, even machine code and assembly are abstractions. Abstractions aren't reality (the map is not the territory and all that), but they capture some aspect of reality, in some sense, and if that aspect happens to be the part you care about, they're useful. It's not about not using abstractions, it's about choosing the correct abstraction for the given problem domain.

Frost1x 7y ago
Abstractions should also simplify. If an abstraction makes something unnecessarily complex, then using abstraction for the sake of abstraction isn't useful. In the spirit of YAGNI, you shouldn't try planning too far ahead with your abstractions. From my experience, sometimes your clever planning all falls into place, but most the time something significant shifts in client demands and your abstractions don't make much sense anymore and add nothing but a layer of complexity to deal with that didn't need to be there to begin with.

icxa 7y ago

The argument against ORMs aren't that they are a leaky abstraction (although they most definitely are). The argument is although they appear to offer you value up front, they cost you much more down the road. The moment you have a hot path query that needs optimizing, you are dropping down to your ORM's "raw sql" mode. Then you do it again. Then again. Then you are ripping out the ORM and spending cycles refactoring it out of your code and replacing it with simpler abstractions. I always find the people who don't believe writing raw SQL is preferable to using an ORM is usually down to a lack of experience or because they have been coerced to use ORMs by their "enterprise grade" language (usually your C# developers of the world, no offense to you all but if every time you look up examples of data operations it's dealing with EntityFramework you are probably going to wind up with a lot of devs using EntityFramework). The ORMers, as I call them, don't have confidence in their own SQL ability, and they haven't experienced the aforementioned situation enough times to realize you are better off starting with SQL to begin with. The realization is, if I could summarize it: yes all abstractions are at least somewhat leaky, so you are better to use simpler ones than complex ones (and if you need something more complex, compose it from simpler ones)

jtdev 7y ago
This is reflective of my experience with the ORM zealots as well... SQL is already an abstraction, and it happens to be one of the most well thought out, battle tested abstractions we have in the software world... it drives me nuts when junior devs and devs with poor SQL abilities want to pollute a codebase with EF, Hibernate, etc. without understanding if it’s necessary or even helpful beyond simply allowing the developer to continue onward without spending time becoming proficient in SQL. That said, there are scenarios where an ORM is helpful, but I personally find those occasions to be the exception.
- ajross 7y ago
  > SQL is already an abstraction Still a leaky one, though. Sure, the syntax likes to pretend that you "just describe what you want", but in practice getting queries on large data sets to perform well involves layers of optimization, all the way down to deciding on how data is partitioned between storage backends and what the on-disk format of it and its indexes needs to be. In fact I view SQL as a somewhat thin abstraction, with a bunch of unavoidable (or at least unavoided) holes. Maybe that's part of the reason it's been so successful.

Nasrudith 7y ago

Really I think the point is to know what you are abstracting for and not simply dogmatism abstraction for the sake of abstraction or "purity". If you expect to have a system which handles the low level details that is fine. If you don't want it to possible to mess up state consistency at the cost of possible performance fine. However, if you expect abstraction to mean latency doesn't vary at all regardless of if it is on chip cache, RAM, hard drive you are going to have a bad time unless you are dealing with a library that specifically tries to mask timing attacks.

jrochkind1 7y ago

Indeed software engineering pretty much consists of abstractions. (And computer science is basically the science of formalized abstraction, that's basically what CS is). If we didn't have any abstractions at all.. we'd all be writing machine code in 1s and 0s. (and even that is an abstraction, of course...) That wouldn't get us very far. People, including in this thread, also often talk about a particular abstraction being "a leaky abstraction". Implying some abstractions are, some aren't. Whereas in fact the 'law' quoted says that all "non-trivial abstractions, to some degree, are leaky." They're all leaky, the map is never the territory. But it's the business we're in. (But some abstractions are more useful, more successful, and/or more dangerous than others... in a given context of usage, always in a given context).

alexandercrohde 7y ago
I agree all engineering is abstractions. Every function, data-structure, program. >> [People] talk about a particular abstraction being "a leaky abstraction". Implying some abstractions are, some aren't. There are degrees of leakiness though, which matter a great deal. The concept of a "file" is an abstraction, instead we could all write direct to disk sectors. But its value outweighs the leak associated with it. Other abstractions aren't always so useful, sometimes a coworker makes a class that's riddled with exceptions and caveats.
- jrochkind1 7y ago
  Some abstractions are definitely more useful/successful/robust than others. (I think I want to say usually for a particular context, not as a universal/existential thing. Sometimes that context grows to be everyone's... we all use a 'file' abstraction because it was so useful the entire field adopted it, and stopped considering alternative abstractions, and everyone takes as a starting assumption to work within the context where 'file' is useful (except in extreme edge cases maybe) -- for better or worse.) Deciding which is which, whether a particular abstraction's value outweighs the "leak" in particular contexts or in "general" -- well, that's what endless internet debate is made of. It's usually not clear as a consensus until quite some time has passed (and even then heretics can challenge it). I agree it's the right question, but it can't be answered simply by saying "leaky abstraction".

cletus 7y ago

They (and maybe you?) are missing the point of the Law of Leaky Abstractions here. The point is that no matter how good your abstraction there will be corner cases where it gets in the way. This is also a great example of the perfect being an enemy of the good. Some people will try to get that last gap out of their abstraction to the point of it then becoming a little bit worse in all cases and still not being perfect. The point is that your abstraction shouldn't get in your way. It should be easy to bypass when you need to. The problem with ORMs in particular is that this tends to be impossible. Any ORM I've used has added a caching layer so then you can't run raw SQL queries and the ORM is never as expressive enough as raw SQL. Sometimes you just want a subset of columns but ORMs don't tend to support that since a table is mapped to an object and it's either all or nothing. Years ago, this is why I eschewed the likes of Hibernate and preferred things like iBatis.

tomnipotent 7y ago
> Sometimes you just want a subset of columns There isn't a mature ORM I know of that doesn't support this, as either partial bindings to the mapped object (with empty properties) or whatever k/v primitive is supported by the language.

barbecue_sauce 7y ago

I think `fetch` is the real leaky abstraction.

Bodell 7y ago

To agree on this inside of a broader context: I would argue that there are so many abstractions in language (“good”, “evil”, “god”, even less lofty things such as “color”, “heat”, “weather”), science ("gravity" would probably an arguable but interesting example as well as one that came up recently with a neighbor of mine), and even basic functional things (including “plumbing”, “electricity”, and even “computers” at quite literally every level of understanding above mathematics), so in keeping all of that in mind possibly our entire sense of being and living in the world would be completely debilitating without making use of an innumerable amount of abstractions. That said there are always valid arguments against the use of particular abstractions, as they can be seen in practice as being largely more detrimental that helpful. Many make this argument against "god". My High school physics teacher made the argument against the use of “centripetal force.” I myself like to remind people that they probably know far less about the world around them than they think simply because that have replaced conceptual knowledge with a definition and categorical knowledge. Though at the end of the day, I feel that abstractions in general are quite harmless, as long as you are aware of the fact that they are just that “abstractions.”

hyeomans 7y ago

I would rather see the first one. I'm no expert on software development, however I've seen that as time progresses new requirements are added. Cross-cutting requirements are specially hard to implement on your second option, imagine a request id has to be sent. My 2c anyway