3 ms·

One solution is to not use an ORM. These problems are framework problems. I find the cognitive overhead of worrying whether a framework might do the wrong thin

One solution is to not use an ORM. These problems are framework problems. I find the cognitive overhead of worrying whether a framework might do the wrong thing or diagnosing problems after they arise often override any cost savings frameworks provide in the first place.

9dev 2y ago
I totally share your frustrations over ORMs and the issues they cause, but I seriously doubt your cost calculations. The sheer number of working CRUD applications that don’t rely on fancy queries is mind-boggling, and most developers engaging in high-level system design seem mostly oblivious to this. The ORM may not make for pretty or efficient queries, but it enables armies of mediocre programmers to crunch out generic line of business applications very fast. The performance of these apps may not be what it could be, but while we are still busy discussing the finer details of query planner optimisation, folks are making a fortune with N+1 queries in production that virtually nobody cares about.
- ta2234234242 2y ago
  > but it enables armies of mediocre programmers to crunch out generic line of business applications very fast. Or good programmers that see writing raw SQL as a premature optimization. Someone on here once said that often companies end up reinventing their own ORM once the number of queries they have to maintain get up into the thousands. But they've usually made a really shitty ORM by that point in time.
Daishiman 2y ago
Not using an ORM is a solution in the same way that the solution to changing a car's oil is to not own a car. I've spent a good 20 years building web apps. The speed and productivity of having an ORM like Django's a well-integrated into business logic is unparalleled with the possible exception of Rails and Laravel, maybe. Systems that have 50+ tables become _trivial_ to maintain by one person. You can refactor an underlying database table without having to rewrite queries. There are entire classes of refactoring that go away with the ORM. Serving database objects with Django Rest Framework or building up CRUD interfaces with the Django admin would simply be impossible otherwise. Getting the hang of the ORM took me all of 6 months to build up to the nastiest of edge cases, which are functionally as difficult to debug as any complex SQL query with window functions or esoteric functions. The discourse is irrationally framed towards the first months of learning, for a technology that I've been using for 14 years with _no_ sign of finding a replacement.
- henryfjordan 2y ago
  But now I need a developer with >6months of experience in that particular ORM to maintain my app, instead of just someone who knows SQL. Nothing is free.
  - Daishiman 2y ago
    You would have needed someone to get up to speed in all the things you had to reinvent that frameworks already did for you, except that frameworks do it better due to the thousands of eyeballs that have looked at the same problems over and over again.
deleted 2y ago
[deleted]
ljm 2y ago
You’ll get N+1 or quadratic queries whether or not you use an ORM. At the end of the day you are allocating time on the database and it doesn’t care about the way you build the query. Runtime compute tends to be a lot cheaper than DB compute too - might as well run those cycles before the DB does.
- henryfjordan 2y ago
  If you actually have to reckon with the SQL queries in your app, N+1 becomes less likely. Consider `rows.map { row -> repository.fetchRelatedRow(row.id) }`. That's much more obviously going to cause issues than `rows.map { row.relatedRow }`. You don't need to cross-reference the model code for row to see if relatedRow is eager loaded or not, the answer is right there for you.
- rand_r 2y ago
  It is true that DB compute is relatively expensive, but doing a join on the DB itself is obscenely more efficient vs doing it manually in a nested loop on an app server. Also, the DB is generally a lot smarter than doing quadratic operations. The usual worst case is a Merge Join which involves sorting both tables first, to do an efficient join in nlogn time. It’s just not feasible to compete with the DB for efficiency without reimplementing a DB, which is pretty yikes. See https://www.cybertec-postgresql.com/en/join-strategies-and-performance-in-postgresql/ https://www.cybertec-postgresql.com/en/join-strategies-and-p...
- switchbak 2y ago
  I don't get that. If you're not using an ORM, you're simply looking at writing the queries to satisfy your requirements - nothing is guiding you towards an N+1 path. There's nothing inherent about how we access data in a relational world that means we have to adopt such an anti-pattern. It does mean that you have to think ahead a little bit though.

9dev 2y ago

I totally share your frustrations over ORMs and the issues they cause, but I seriously doubt your cost calculations. The sheer number of working CRUD applications that don’t rely on fancy queries is mind-boggling, and most developers engaging in high-level system design seem mostly oblivious to this. The ORM may not make for pretty or efficient queries, but it enables armies of mediocre programmers to crunch out generic line of business applications very fast. The performance of these apps may not be what it could be, but while we are still busy discussing the finer details of query planner optimisation, folks are making a fortune with N+1 queries in production that virtually nobody cares about.

ta2234234242 2y ago
> but it enables armies of mediocre programmers to crunch out generic line of business applications very fast. Or good programmers that see writing raw SQL as a premature optimization. Someone on here once said that often companies end up reinventing their own ORM once the number of queries they have to maintain get up into the thousands. But they've usually made a really shitty ORM by that point in time.

Daishiman 2y ago

Not using an ORM is a solution in the same way that the solution to changing a car's oil is to not own a car. I've spent a good 20 years building web apps. The speed and productivity of having an ORM like Django's a well-integrated into business logic is unparalleled with the possible exception of Rails and Laravel, maybe. Systems that have 50+ tables become _trivial_ to maintain by one person. You can refactor an underlying database table without having to rewrite queries. There are entire classes of refactoring that go away with the ORM. Serving database objects with Django Rest Framework or building up CRUD interfaces with the Django admin would simply be impossible otherwise. Getting the hang of the ORM took me all of 6 months to build up to the nastiest of edge cases, which are functionally as difficult to debug as any complex SQL query with window functions or esoteric functions. The discourse is irrationally framed towards the first months of learning, for a technology that I've been using for 14 years with _no_ sign of finding a replacement.

henryfjordan 2y ago
But now I need a developer with >6months of experience in that particular ORM to maintain my app, instead of just someone who knows SQL. Nothing is free.
- Daishiman 2y ago
  You would have needed someone to get up to speed in all the things you had to reinvent that frameworks already did for you, except that frameworks do it better due to the thousands of eyeballs that have looked at the same problems over and over again.

deleted 2y ago

[deleted]

ljm 2y ago

You’ll get N+1 or quadratic queries whether or not you use an ORM. At the end of the day you are allocating time on the database and it doesn’t care about the way you build the query. Runtime compute tends to be a lot cheaper than DB compute too - might as well run those cycles before the DB does.

henryfjordan 2y ago
If you actually have to reckon with the SQL queries in your app, N+1 becomes less likely. Consider `rows.map { row -> repository.fetchRelatedRow(row.id) }`. That's much more obviously going to cause issues than `rows.map { row.relatedRow }`. You don't need to cross-reference the model code for row to see if relatedRow is eager loaded or not, the answer is right there for you.
rand_r 2y ago
It is true that DB compute is relatively expensive, but doing a join on the DB itself is obscenely more efficient vs doing it manually in a nested loop on an app server. Also, the DB is generally a lot smarter than doing quadratic operations. The usual worst case is a Merge Join which involves sorting both tables first, to do an efficient join in nlogn time. It’s just not feasible to compete with the DB for efficiency without reimplementing a DB, which is pretty yikes. See https://www.cybertec-postgresql.com/en/join-strategies-and-performance-in-postgresql/ https://www.cybertec-postgresql.com/en/join-strategies-and-p...
switchbak 2y ago
I don't get that. If you're not using an ORM, you're simply looking at writing the queries to satisfy your requirements - nothing is guiding you towards an N+1 path. There's nothing inherent about how we access data in a relational world that means we have to adopt such an anti-pattern. It does mean that you have to think ahead a little bit though.