5 ms·

Django Rest Framework general model serializer

64 points by norbert_mate 12y ago

kayoone 12y ago
I am quite amazed this has to be done manually in Django. I work a lot with Symfony2 in PHP and that kind of Object serialization thing is pretty much built in. I just return my model from the controller and it gets serialized to JSON including child objects with pretty fine grained controls about what to serialize. Besides, after working a lot with REST webservices and AngularJS, i don't know if serializing child objects is really a good idea vs just getting the id and then using the api endpoint for the specific children separately. Having child objects serialized creates problems when doing PUT/UPDATE requests and you can also end up with fairly large sets of JSON. Of course doing dozens of requests instead (especially for lists) is also not the best of ideas, so i am not really sure how others handle this ?
- norbert_mate 12y ago
  In Django Rest Framework (DRF) you need serializer class. Maybe this is because DRF is not tightly coupled with the models from Django framework. As far as I know DRF does not support PUT/UPDATE operations for nested serializers. But there are cases when you just need to read. Like getting an activity feed which can't be modified directly by the users. I think if you have a good caching system it's better to return all the data for one request instead of creating new requests to get details about each entity.
- andybak 12y ago
  Django does this out of the box: https://docs.djangoproject.com/en/1.7/topics/serialization/ https://docs.djangoproject.com/en/1.7/topics/serialization/ But DRF adds lots of extra functionality. Even with DRF it's as simple as adding: class UserSerializer(serializers.ModelSerializer): class Meta: model = User to get full automatic serialization of a model. > I don't know if serializing child objects is really a good idea Depends on the specific case. For internal APIs I tend to create endpoints that give a view what it needs in a single request.
  - norbert_mate 12y ago
    Correct. It's very easy to create a simple model serializer and this is what the general model serializer does. It's no big deal just saves a few line of code.
  - whalesalad 12y ago
    You say "out of the box" but it's not that simple... 'blah blah DateTime is not JSON serializable'
    - andybak 12y ago
      Just tried it: from django.core import serializers print serializers.serialize("json", StockCount.objects.all()[:2]) >> [{"fields": {"timestamp": "1900-01-02T00:00:00Z", "version": 0, "item_instance": null, "item_model": 408, "stock_level": 999, "location": 2}, "model": "stocklevels.stockcount", "pk": 1}, {"fields": {"timestamp": "1900-01-02T00:00:00Z", "version": 0, "item_instance": null, "item_model": 409, "stock_level": 999, "location": 2}, "model": "stocklevels.stockcount", "pk": 2}]
- numlocked 12y ago
  We tend to load things (GET) in one large request that pulls a big hunks of JSON for everything certain parts of the app need. These have lots of nested serialization going on. We then have separate, traditional REST endpoints for updates/creates and incremental updates of the larger object.
- adambratt 12y ago
  No one has answered your question properly yet. This is an edge case for something called Django Content Types. It's basically a way to handle some sort of polymorphism in a table. Let's say you have a table called activity feed like in this example. Whether the items in the feed are images, blog posts, or status updates they're all going to have something in common and you'll have endpoints that should link any relevant tables and combine them. So you'd have an activity table and it would have a column for content_type and then content_object_id. Content type is basically just a reference to another DB table (blog or status update) with more info on this activity and content_object_id is the actual id of the row in that table. I've used Symfony, Laravel, and pretty much every other trendy PHP framework and nothing comes close to the ease of building APIs that is inherent in Django Rest Framework. This is just a very specific use case that you normally have to handle using it's own custom class. This just shows a way you can have DRF serialize it like a normal object without issue.
- lazerscience 12y ago
  There's not a general solution to serializing child objects, it totally depends on the use case and the number of children. So for example if you have a lot of child objects it might not even make sense to include theirs ids, but have a seperate (maybe nested) resource to retrieve them. On the other hand if there are only few children you might want to include them in their parent resource.
skatenerd 12y ago
I really have a problem with the concept of the Serializer in DRF. It handles so much that has nothing to do with serialization. It performs something like de-serialization, where it takes an incoming dictionary and turns it into an ORM object. The docs even recommend using its ".save()" method. Save? on a Serializer? I think it's a really cool concept if you think of it as more of a Lens. The so-called serializer lets you declare properties of "fields", and these properties will influence both the native-to-ORM conversion, as well as the ORM-to-native conversion.
- norbert_mate 12y ago
  I agree with you. It is a bit weird that a serializer can interact with the database but what it basically does: hides the database interaction.
  - skatenerd 12y ago
    out of curiosity, how often do you find yourself reading the DRF source code to figure out whats happening?
    - norbert_mate 12y ago
      "The best documentation is the source code" :-) . I use to read it but sometimes is really hard to understand what's going on. As you said the serializer does a lot of thing.
  - tomchristie 12y ago
    The model / model manager layer is the right place at which to design your state changing API. REST framework absolutely you to work with the grain there. Some good practice I'd recommend... * Write the `create()` and/or `update()` methods explicitly on the serializer class. * Push logic into the model and model manager where possible and only have the serializer `.save()` as a thin layer on top of that. That way a serializer class still has all the behavior it needs to map both ways between persisted objects and their corresponding native python representations, but you still have a well separated model API.
numlocked 12y ago
I think it's somewhat rare to have generic relations (content type + model id) popping up this way in serializers. I tend to use generic FKs for things like "tags" or or "comments" that can apply to multiple objects. As a result, I'm almost always requesting the parent object ("post") and want the model with the generic FK serialized in that response ("post.comments"), not the other way around. Just looking through largish DRF codebase, I'm not sure there are many places I could use this, thought it seems like a good idea if your code has a lot of these scenarios.
- norbert_mate 12y ago
  Thanks.
mercnet 12y ago
For anyone looking for argument parsing and serialization with Flask, I recommend webargs (http://webargs.readthedocs.org/ http://webargs.readthedocs.org/), marshmallow (http://marshmallow.readthedocs.org http://marshmallow.readthedocs.org), and flask-marshmallow (http://flask-marshmallow.readthedocs.org/ http://flask-marshmallow.readthedocs.org/).
- ddorian43 12y ago
  what rest-framework do you use with flask ? or build one yourself using what components (except these 2)?
  - plq 12y ago
    You can use Spyne[1]'s HttpRpc and JsonDocument protocols via Flask-Spyne[2]. You can actually use Spyne with Django as well[3] (and Spyne has a "general model serializer" that can be used with a lot less boilerplate than what's shown here). Spyne supports a lot of protocols as well as persisting to a relational database via SQLAlchemy. Disclaimer: I'm the author of spyne. [1]: http://spyne.io http://spyne.io [2]: https://pypi.python.org/pypi/Flask-Spyne https://pypi.python.org/pypi/Flask-Spyne [3]: https://github.com/arskom/spyne/tree/master/examples/django/rpctest https://github.com/arskom/spyne/tree/master/examples/django/...
languagehacker 12y ago
Nice to see Django Rest Framework getting some press, but I don't understand how this provides value beyond the existing documentation on readthedocs.
- norbert_mate 12y ago
  Hi languagehacker, this is not science fiction. It just shows you a way of implementing a serializer that works for all kind of models in Django Rest Framework.
languagehacker 12y ago
Nice to see Django Rest Framework getting some press, but I don't understand how this provides value beyond the existing documentation on readthedocs.

64 points by norbert_mate 12y ago

I am quite amazed this has to be done manually in Django. I work a lot with Symfony2 in PHP and that kind of Object serialization thing is pretty much built in. I just return my model from the controller and it gets serialized to JSON including child objects with pretty fine grained controls about what to serialize. Besides, after working a lot with REST webservices and AngularJS, i don't know if serializing child objects is really a good idea vs just getting the id and then using the api endpoint for the specific children separately. Having child objects serialized creates problems when doing PUT/UPDATE requests and you can also end up with fairly large sets of JSON. Of course doing dozens of requests instead (especially for lists) is also not the best of ideas, so i am not really sure how others handle this ?

norbert_mate 12y ago
In Django Rest Framework (DRF) you need serializer class. Maybe this is because DRF is not tightly coupled with the models from Django framework. As far as I know DRF does not support PUT/UPDATE operations for nested serializers. But there are cases when you just need to read. Like getting an activity feed which can't be modified directly by the users. I think if you have a good caching system it's better to return all the data for one request instead of creating new requests to get details about each entity.
andybak 12y ago
Django does this out of the box: https://docs.djangoproject.com/en/1.7/topics/serialization/ https://docs.djangoproject.com/en/1.7/topics/serialization/ But DRF adds lots of extra functionality. Even with DRF it's as simple as adding: class UserSerializer(serializers.ModelSerializer): class Meta: model = User to get full automatic serialization of a model. > I don't know if serializing child objects is really a good idea Depends on the specific case. For internal APIs I tend to create endpoints that give a view what it needs in a single request.
- norbert_mate 12y ago
  Correct. It's very easy to create a simple model serializer and this is what the general model serializer does. It's no big deal just saves a few line of code.
- whalesalad 12y ago
  You say "out of the box" but it's not that simple... 'blah blah DateTime is not JSON serializable'
  - andybak 12y ago
    Just tried it: from django.core import serializers print serializers.serialize("json", StockCount.objects.all()[:2]) >> [{"fields": {"timestamp": "1900-01-02T00:00:00Z", "version": 0, "item_instance": null, "item_model": 408, "stock_level": 999, "location": 2}, "model": "stocklevels.stockcount", "pk": 1}, {"fields": {"timestamp": "1900-01-02T00:00:00Z", "version": 0, "item_instance": null, "item_model": 409, "stock_level": 999, "location": 2}, "model": "stocklevels.stockcount", "pk": 2}]
numlocked 12y ago
We tend to load things (GET) in one large request that pulls a big hunks of JSON for everything certain parts of the app need. These have lots of nested serialization going on. We then have separate, traditional REST endpoints for updates/creates and incremental updates of the larger object.
adambratt 12y ago
No one has answered your question properly yet. This is an edge case for something called Django Content Types. It's basically a way to handle some sort of polymorphism in a table. Let's say you have a table called activity feed like in this example. Whether the items in the feed are images, blog posts, or status updates they're all going to have something in common and you'll have endpoints that should link any relevant tables and combine them. So you'd have an activity table and it would have a column for content_type and then content_object_id. Content type is basically just a reference to another DB table (blog or status update) with more info on this activity and content_object_id is the actual id of the row in that table. I've used Symfony, Laravel, and pretty much every other trendy PHP framework and nothing comes close to the ease of building APIs that is inherent in Django Rest Framework. This is just a very specific use case that you normally have to handle using it's own custom class. This just shows a way you can have DRF serialize it like a normal object without issue.
lazerscience 12y ago
There's not a general solution to serializing child objects, it totally depends on the use case and the number of children. So for example if you have a lot of child objects it might not even make sense to include theirs ids, but have a seperate (maybe nested) resource to retrieve them. On the other hand if there are only few children you might want to include them in their parent resource.

skatenerd 12y ago

I really have a problem with the concept of the Serializer in DRF. It handles so much that has nothing to do with serialization. It performs something like de-serialization, where it takes an incoming dictionary and turns it into an ORM object. The docs even recommend using its ".save()" method. Save? on a Serializer? I think it's a really cool concept if you think of it as more of a Lens. The so-called serializer lets you declare properties of "fields", and these properties will influence both the native-to-ORM conversion, as well as the ORM-to-native conversion.

norbert_mate 12y ago
I agree with you. It is a bit weird that a serializer can interact with the database but what it basically does: hides the database interaction.
- skatenerd 12y ago
  out of curiosity, how often do you find yourself reading the DRF source code to figure out whats happening?
  - norbert_mate 12y ago
    "The best documentation is the source code" :-) . I use to read it but sometimes is really hard to understand what's going on. As you said the serializer does a lot of thing.
- tomchristie 12y ago
  The model / model manager layer is the right place at which to design your state changing API. REST framework absolutely you to work with the grain there. Some good practice I'd recommend... * Write the `create()` and/or `update()` methods explicitly on the serializer class. * Push logic into the model and model manager where possible and only have the serializer `.save()` as a thin layer on top of that. That way a serializer class still has all the behavior it needs to map both ways between persisted objects and their corresponding native python representations, but you still have a well separated model API.

numlocked 12y ago

I think it's somewhat rare to have generic relations (content type + model id) popping up this way in serializers. I tend to use generic FKs for things like "tags" or or "comments" that can apply to multiple objects. As a result, I'm almost always requesting the parent object ("post") and want the model with the generic FK serialized in that response ("post.comments"), not the other way around. Just looking through largish DRF codebase, I'm not sure there are many places I could use this, thought it seems like a good idea if your code has a lot of these scenarios.

norbert_mate 12y ago
Thanks.

mercnet 12y ago

For anyone looking for argument parsing and serialization with Flask, I recommend webargs (http://webargs.readthedocs.org/ http://webargs.readthedocs.org/), marshmallow (http://marshmallow.readthedocs.org http://marshmallow.readthedocs.org), and flask-marshmallow (http://flask-marshmallow.readthedocs.org/ http://flask-marshmallow.readthedocs.org/).

ddorian43 12y ago
what rest-framework do you use with flask ? or build one yourself using what components (except these 2)?
- plq 12y ago
  You can use Spyne[1]'s HttpRpc and JsonDocument protocols via Flask-Spyne[2]. You can actually use Spyne with Django as well[3] (and Spyne has a "general model serializer" that can be used with a lot less boilerplate than what's shown here). Spyne supports a lot of protocols as well as persisting to a relational database via SQLAlchemy. Disclaimer: I'm the author of spyne. [1]: http://spyne.io http://spyne.io [2]: https://pypi.python.org/pypi/Flask-Spyne https://pypi.python.org/pypi/Flask-Spyne [3]: https://github.com/arskom/spyne/tree/master/examples/django/rpctest https://github.com/arskom/spyne/tree/master/examples/django/...

languagehacker 12y ago

Nice to see Django Rest Framework getting some press, but I don't understand how this provides value beyond the existing documentation on readthedocs.

norbert_mate 12y ago
Hi languagehacker, this is not science fiction. It just shows you a way of implementing a serializer that works for all kind of models in Django Rest Framework.

languagehacker 12y ago

Nice to see Django Rest Framework getting some press, but I don't understand how this provides value beyond the existing documentation on readthedocs.