10 ms·

I'm afraid it is a major problem in practice - I'm didn't write the post in the abstract I wrote it because this problem comes up over and over. If you've neve

I'm afraid it is a major problem in practice - I'm didn't write the post in the abstract I wrote it because this problem comes up over and over. If you've never seen it then lucky you! :) Personally the conclusion I'm coming to is any interface that presents a table as an class is just the wrong affordance and likely to lead to problems down the line. If programmers modeled tables as sets instead of objects we wouldn't have all these problem apps that page the database into program memory while serving a web request.

aurisor 7y ago
It is a major problem in practice. It's called the "N+1" problem and it's exhaustively studied and documented, with mitigation strategies for every ORM that encounters it. This is a bit like acting as if you're the first person to realize that null pointer deferences are a problem and happily telling the entire world to "just switch to pass by value."
- calpaterson 7y ago
  You aren't right about that - the N+1 problem is just one specific issue WRT joins and while the example I give does have it that is not the only thing wrong with it. The rot is more general sadly.
  - aurisor 7y ago
    > ISBNs of all deceased Italian authors Book.joins(:author).where(author: { deceased: true }).map &:isbn` > first class transactions / decrement race condition Book.find(isbn: 1).decrement :copies_in_stock > Row-locking Book.transaction do b = Book.lock("FOR UPDATE NOWAIT").find_(1) # logic around handling out of stock b.copies_in_stock -= 1 b.save! end Those are all of your examples and the Ruby code that implements the solution you want. (Disclaimer: there are probably small syntax errors here, you get the gist) Honestly this article sounds like it was written by someone who read a 10 year old Martin Fowler article and decided to critique it without even a cursory understanding of modern ORMs. Even more strangely, when everyone starts pointing out the obvious fact that ORMs do solve these problems in practice, you've decided to move the goalposts to say "oh, well the straw man abstract pattern doesn't solve for these real-world problems." In the future I'd recommend you familiarize yourself with a problem space before trying to publish on it.
  - sanderjd 7y ago
    I think we are using "problem" in two different ways that are each reasonable but are not the same: 1. something that happens and needs to be fixed, and 2. something that is fundamental and difficult to overcome. In my comment up-thread, I meant more the #2 definition, but you clearly mean the #1 definition. I spent a number of years working on a Rails application that used ActiveRecord and which accrued quite a large set of data. In that job, I fixed a whole bunch of N+1 problems, and wrote sql queries with complex joins before the nicer query-builder arel syntax was implemented. I was a pretty vocal critic of AR at my company because of these sorts of headaches (though I was much happier post-arel). But these were all very surmountable problems, the libraries had the capabilities necessary to fix the problems, and as our team matured in our intuition for our data model and size, we ran into these things less and less. In other words, I see this as very much a #1 type of problem rather than a #2 type of problem. A good analogy seems to be n^2 algorithms. In much the same way, you can write one of those and it works fine for awhile, but then one day you may find that your data has grown enough that it no longer works, and it may be very mysterious what happened. But with some experience and work, you can figure it out and probably fix it. And as you gain experience, you can more wisely decide whether to avoid the n^2 algorithm from the start, or whether it will work fine for your data size.
cryptonector 7y ago
Allow me to chime in and agree with you that in real life there exist real applications using ORMs that have or cause these problems even where the ORMs in question provide modern query facilities that don't suck. Whether this happens because the ORMs were not that advanced when the apps were developed, or the devs were not that knowledgeable, doesn't matter -- the apps exist, and I know because I've had to deal with some such. I agree very much with the idea that the problem is fundamentally caused by wanting to present objects to apps, and that dealing with queries instead is better and the real solution. Of course, the queries can't be SQL -- we can't have that! So we end up with N+1 query languages: SQL and N more non-SQLs. But N+1 query languages is a lot better than no query languages as in earlier ORMs.
- robocat 7y ago
  SQL has hideous security issues with SQL injection using strings, and ActiveRecord avoids that, so SQL is bad and ActiveRecord is good. My statement is just as true and just as disingenuous as Cal’s article. > in real life there exist real applications using ORMs that have or cause these problems even where the ORMs in question provide modern query facilities that don't suck And surely you have also seen plenty of equivalent examples of bad code written using SQL? Cal is attacking an imaginary ActiveRecord pattern, while ignoring the features of the most famous implementation (RoR), and ignoring the benefits of ActiveRecord. Dumb.
  - cryptonector 7y ago
    > SQL has hideous security issues with SQL injection using strings, and ActiveRecord avoids that, so SQL is bad and ActiveRecord is good. My statement is just as true and just as disingenuous as Cal’s article. Most SQL RDBMSes have proper parametrized queries, and there's no excuse not to use them. This is no excuse to not use SQL. It's like saying that no one should use ORMs because you can use them poorly. > And surely you have also seen plenty of equivalent examples of bad code written using SQL? I've seen bad Java, C, C++, Python, Perl, bash, etc... The SQL I work with is no worse than the Java, C, ... that I work with. > Cal is attacking an imaginary ActiveRecord pattern, while ignoring the features of the most famous implementation (RoR), and ignoring the benefits of ActiveRecord. Dumb. He was talking about a particular pattern, not Ruby on Rails.

aurisor 7y ago

It is a major problem in practice. It's called the "N+1" problem and it's exhaustively studied and documented, with mitigation strategies for every ORM that encounters it. This is a bit like acting as if you're the first person to realize that null pointer deferences are a problem and happily telling the entire world to "just switch to pass by value."

calpaterson 7y ago
You aren't right about that - the N+1 problem is just one specific issue WRT joins and while the example I give does have it that is not the only thing wrong with it. The rot is more general sadly.
- aurisor 7y ago
  > ISBNs of all deceased Italian authors Book.joins(:author).where(author: { deceased: true }).map &:isbn` > first class transactions / decrement race condition Book.find(isbn: 1).decrement :copies_in_stock > Row-locking Book.transaction do b = Book.lock("FOR UPDATE NOWAIT").find_(1) # logic around handling out of stock b.copies_in_stock -= 1 b.save! end Those are all of your examples and the Ruby code that implements the solution you want. (Disclaimer: there are probably small syntax errors here, you get the gist) Honestly this article sounds like it was written by someone who read a 10 year old Martin Fowler article and decided to critique it without even a cursory understanding of modern ORMs. Even more strangely, when everyone starts pointing out the obvious fact that ORMs do solve these problems in practice, you've decided to move the goalposts to say "oh, well the straw man abstract pattern doesn't solve for these real-world problems." In the future I'd recommend you familiarize yourself with a problem space before trying to publish on it.
- sanderjd 7y ago
  I think we are using "problem" in two different ways that are each reasonable but are not the same: 1. something that happens and needs to be fixed, and 2. something that is fundamental and difficult to overcome. In my comment up-thread, I meant more the #2 definition, but you clearly mean the #1 definition. I spent a number of years working on a Rails application that used ActiveRecord and which accrued quite a large set of data. In that job, I fixed a whole bunch of N+1 problems, and wrote sql queries with complex joins before the nicer query-builder arel syntax was implemented. I was a pretty vocal critic of AR at my company because of these sorts of headaches (though I was much happier post-arel). But these were all very surmountable problems, the libraries had the capabilities necessary to fix the problems, and as our team matured in our intuition for our data model and size, we ran into these things less and less. In other words, I see this as very much a #1 type of problem rather than a #2 type of problem. A good analogy seems to be n^2 algorithms. In much the same way, you can write one of those and it works fine for awhile, but then one day you may find that your data has grown enough that it no longer works, and it may be very mysterious what happened. But with some experience and work, you can figure it out and probably fix it. And as you gain experience, you can more wisely decide whether to avoid the n^2 algorithm from the start, or whether it will work fine for your data size.

cryptonector 7y ago

Allow me to chime in and agree with you that in real life there exist real applications using ORMs that have or cause these problems even where the ORMs in question provide modern query facilities that don't suck. Whether this happens because the ORMs were not that advanced when the apps were developed, or the devs were not that knowledgeable, doesn't matter -- the apps exist, and I know because I've had to deal with some such. I agree very much with the idea that the problem is fundamentally caused by wanting to present objects to apps, and that dealing with queries instead is better and the real solution. Of course, the queries can't be SQL -- we can't have that! So we end up with N+1 query languages: SQL and N more non-SQLs. But N+1 query languages is a lot better than no query languages as in earlier ORMs.

robocat 7y ago
SQL has hideous security issues with SQL injection using strings, and ActiveRecord avoids that, so SQL is bad and ActiveRecord is good. My statement is just as true and just as disingenuous as Cal’s article. > in real life there exist real applications using ORMs that have or cause these problems even where the ORMs in question provide modern query facilities that don't suck And surely you have also seen plenty of equivalent examples of bad code written using SQL? Cal is attacking an imaginary ActiveRecord pattern, while ignoring the features of the most famous implementation (RoR), and ignoring the benefits of ActiveRecord. Dumb.
- cryptonector 7y ago
  > SQL has hideous security issues with SQL injection using strings, and ActiveRecord avoids that, so SQL is bad and ActiveRecord is good. My statement is just as true and just as disingenuous as Cal’s article. Most SQL RDBMSes have proper parametrized queries, and there's no excuse not to use them. This is no excuse to not use SQL. It's like saying that no one should use ORMs because you can use them poorly. > And surely you have also seen plenty of equivalent examples of bad code written using SQL? I've seen bad Java, C, C++, Python, Perl, bash, etc... The SQL I work with is no worse than the Java, C, ... that I work with. > Cal is attacking an imaginary ActiveRecord pattern, while ignoring the features of the most famous implementation (RoR), and ignoring the benefits of ActiveRecord. Dumb. He was talking about a particular pattern, not Ruby on Rails.