3 ms·

The Case of the Unusable Django Reusable

13 points by apgwoz 17y ago

mdasen 17y ago
This actually highlights a small problem with the Django ORM: outer joins. select_related() is great when you want to follow a foreign key, but it doesn't work on reverse foreign keys or many to many relationships. So, if you have a blogging app with entries and comments, you can't do Entry.objects.select_related('comment_set'). However, it does work the other way (Comment.objects.select_related('entry')). Basically, the issue is that when you're doing multiple multi-value lookups, making them into objects becomes difficult because of the rows that SQL returns. Let's say that you were trying to look up both the comments and tags at the same time. You'd get: Entry--Comment--Tag 1 sam blue 1 sam red 1 sam green 1 mat blue 1 mat red 1 mat green 1 jon blue 1 jon red 1 jon green 2 adam fun 2 adam party Turning that into an object isn't trivial because you don't have the comment from "sam" three times. It's only there once, but appears in the results set three times because you get join1rows * join2rows in terms of the number of rows returned. And that's an example of two reverse foreign keys with minimal related data. The issue is that it would be easy to shoot one's self in the foot with even more joins. Rails used to do outer joins and then choke (go really slow) if you did something really complex. It now uses an IN() query when you hit the first accessor. I'd still like to see Django support this case as it's a decently common thing to want to do with an application. At the very least, select_related() should raise an error when you try to specify a reverse foreign key that it won't follow. Many people think it follows reverse foreign keys because it gives no indication that it doesn't unless you look at the queries that it's running.
- apgwoz 17y ago
  In the case of a blog though, there aren't many instances when you pull all the comments for every post. Tags, sure, but the most you normally need for comments is the count. There's two ways to approach this. 1) you use extra, or the new aggregate framework (that I haven't tested out) 2) you keep track of a comment_count in your Entry model and probably create a post_save callback on your Comment model. The tags on the other hand are much more difficult, and I haven't figured out a great way to handle this except for saving a pre-rendered version of an entry, or saving a cached version of the tags as JSON (in addition to actually relating them) in the Entry instance. But, this of course means you need to jump through some other hoops to prepare this. And all these hoops really speaks to ORMs not being very effective in all cases.

13 points by apgwoz 17y ago

This actually highlights a small problem with the Django ORM: outer joins. select_related() is great when you want to follow a foreign key, but it doesn't work on reverse foreign keys or many to many relationships. So, if you have a blogging app with entries and comments, you can't do Entry.objects.select_related('comment_set'). However, it does work the other way (Comment.objects.select_related('entry')). Basically, the issue is that when you're doing multiple multi-value lookups, making them into objects becomes difficult because of the rows that SQL returns. Let's say that you were trying to look up both the comments and tags at the same time. You'd get: Entry--Comment--Tag 1 sam blue 1 sam red 1 sam green 1 mat blue 1 mat red 1 mat green 1 jon blue 1 jon red 1 jon green 2 adam fun 2 adam party Turning that into an object isn't trivial because you don't have the comment from "sam" three times. It's only there once, but appears in the results set three times because you get join1rows * join2rows in terms of the number of rows returned. And that's an example of two reverse foreign keys with minimal related data. The issue is that it would be easy to shoot one's self in the foot with even more joins. Rails used to do outer joins and then choke (go really slow) if you did something really complex. It now uses an IN() query when you hit the first accessor. I'd still like to see Django support this case as it's a decently common thing to want to do with an application. At the very least, select_related() should raise an error when you try to specify a reverse foreign key that it won't follow. Many people think it follows reverse foreign keys because it gives no indication that it doesn't unless you look at the queries that it's running.

apgwoz 17y ago
In the case of a blog though, there aren't many instances when you pull all the comments for every post. Tags, sure, but the most you normally need for comments is the count. There's two ways to approach this. 1) you use extra, or the new aggregate framework (that I haven't tested out) 2) you keep track of a comment_count in your Entry model and probably create a post_save callback on your Comment model. The tags on the other hand are much more difficult, and I haven't figured out a great way to handle this except for saving a pre-rendered version of an entry, or saving a cached version of the tags as JSON (in addition to actually relating them) in the Entry instance. But, this of course means you need to jump through some other hoops to prepare this. And all these hoops really speaks to ORMs not being very effective in all cases.