3 ms·
I'm currently in the middle of a very similar situation. It's fine for dev, but throwing hundreds of queries at a single request just isn't going to scale. We
by shabble 14y ago
I'm currently in the middle of a very similar situation. It's fine for dev, but throwing hundreds of queries at a single request just isn't going to scale.
We've ended up with stuff like:
class Meta(SomeResource.Meta):
cls = Challenge
select_related_fields = [
'category', 'owner', 'company', 'bonus', 'reward',
]
prefetch_related_fields = [
'specialities', 'countries',
'companies', 'products',
'groups',
]
queryset = (cls.objects
.select_related(*select_related_fields)
.prefetch_related(*prefetch_related_fields)
.all())
which takes some of the pain away, but going through and profiling exactly which prefetches & joins help is a major hassle.
Resources with relations with full=True are also brutal. Manually stuffing only the required extra data speeds things up greatly, but there's too much boilerplate for my liking. Having some level in-between 'full' and 'resource-uri only' would be nice, but I haven't seen any nice way to do it.
- izak30 14y agoThese are the problems you have with any Django app as you learn to scale it, they're not inherent to Tastypie. They're why prefetch_related and select_related exist, you're Doing Things Right.
- jaytaylor 14y agoThis isn't Right, IMO this is a sloppy mess of a way to develop a scalable API, and only lets you barely eek by. There is no joy.
- izak30 14y agoI wish that I had a better response for this. I don't. Scaling is hard. It's work, It's not usually magic and you have to understand a lot about your system to get it right. Peppering in a few cache statements and figuring out the proper querysets isn't that bad. I know that I can't be seen as an impartial observer in this situation since I commit to Tastypie, but I got involved with the project because it did this stuff right. If you've got any suggestions on improvements I'm all ears and my email is in my profile
- dev360 14y ago+1 - couldnt agree more.