6 ms·

Tips for a new Django developer

126 points by niyazpk 17y ago

teye 17y ago
Great tips. When I was starting, some of these didn't occur to me and the others I knew I should be doing but didn't know how.
ivankirigin 17y ago
This is some really useful stuff. I learned a lot of this over time, and wished I had known it sooner. The point about schema evolution is important. I've taken to using a pickled python object in a text field in every model. That flexibility to store arbitrary data in a hash is amazing. I've used it to, for example, represent twitter accounts that are oauthed, with password, following the business account, or just nothing special. 4 kinds of accounts, with logic in the model to deal with it properly, all without evolving the schema. I only started with the first 2, and added the latter two without a problem. I'm pushing this a bit further now, and transitioning away from models altogether, and experimenting with a non-relational database like couchdb. I'm debating moving to google app engine too, but found some details (like getting on your own domain) to be pretty annoying.
- seiji 17y ago
  > I've taken to using a pickled python object in a text field in every model. That's basically what App Engine does right there. It stores by an id, then the value is your object serialized as a protocol buffer. The App Engine datastore allows you to look up by other properties than id because it indexes every field of your object in the background without you having to deal with it. You can query by pair-wise or more-wise indexes too, but you have to create those manually.
- stanleydrew 17y ago
  Storing pickled python objects is fine as long as you don't care about relational data. But if you aren't careful, data integrity will probably suffer over time as the complexity of your application increases. And that is not good. Anyway I'm sure a lot of people here know this stuff already. Just do your research and make sure the route you go makes sense for your application and your scaling requirements.
- matclayton 17y ago
  Really don't like the sound of that, pickling allows you to serialize any python code, and consequently deserialize it. Presumably you are placing user submitted data into this pickle and therefore without careful sanity checking wouldn't this potentially allow the user to inject code which may be run on deserialization? Someone else may be able to shed more light on this and correct me (please do!) but thats my concern and would love to hear if it was a valid one or not.
  - limmeau 17y ago
    It is possible to build a poisonous pickle which, on depickling, calls arbitrary standard library functions, so accepting pickles from untrusted source is a No. One may think that if only the internal model of the user input is pickled, and the user only gets to update this internal model through web requests in defined ways, thus never getting to upload pickles or construct arbitrary complicated python objects, it should be relatively safe. However, what remains is that if someone manages to blindly inject SQL, he can inject executable code into your application, instead of just injecting data.
  - patio11 17y ago
    Whoa! I was assuming this worked like doing a JSON grab-bag column in Rails or J2EE, which I am a fan of. Executable code would make me much, muuuuuuuuuch cooler on the idea. Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted?
    - RyanMcGreal 17y ago
      > Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted? Sure. You can even extend the `json` module to serialize custom data types not defined in the JSON spec.
  - ivankirigin 17y ago
    No, I'm just serializing dicts. Any user input is already in a string, so sanitized. I also usually make helper functions to get/set things in the dict, so I don't operate on the field outside the model often. This snippet is really helpful: http://www.djangosnippets.org/snippets/513/ http://www.djangosnippets.org/snippets/513/
- FraaJad 17y ago
  Pickling python objects and storing is an very old practice in Python world. Zope has been persisting objects in ZODB (a giant pickle jar with some transactional features) since 1998.
  - ivankirigin 17y ago
    New to me :) I love it
- banjiewen 17y ago
  Would be interested to hear your take on Django+CouchDB - are you using a library like Couchdbkit, or rolling your own? Are you connecting it to the built-in admin?
  - ivankirigin 17y ago
    Still exploring. I'll post something when it actually works.
hswolff 17y ago
Thank you for this. I am starting and hopefully this'll give me a solid foundation.
rmanocha 17y ago
Another point I would add to this list is to not be afraid of reading the Django Source code. While the docs cover a lot, you can find a lot of hidden gems simply going through the source code - it is highly readable and most stuff is where you would expect it to be. Of-course, be careful while doing so though 'cause the stuff that isn't documented can change at anytime, and may break your code.
xhuang 17y ago
thanks for this, i just started django development couple days ago, and this is very useful. another thing worth mention is that use generic view, it will save lots typing:)
wakeupthedawn 17y ago
These are pretty good. I really liked #6, the tip for template_tags. I hadn't thought of that before and will definitely do that. I will also look into extensions of the template system as suggested at the bottom. Those were really helpful, even for someone who has been using Django for a few months. I disagree with #4, the suggestion to not include business logic in views. What Django calls 'views' are what other frameworks call 'controllers' (MTV=MVC), and the business logic should reside there. I'm totally for separating the code into helper functions in other files, but putting business logic into models definitely violates the idea of MVC. For #5, I would recommend the second approach of having a separate settings.py for production vs development. Or at least code it where DEBUG=False is only overridden if socket confirms you are on a development machine - not the other way around. Debug contains too much info to risk exposing in production, but maybe my fear of socket screwing up is unfounded.
- mhb 17y ago
  For template tags, is doing it that way better than importing them in a base template which is included by the children templates?
  - kennethlove 17y ago
    Templatetags don't inherit (at least not in my experience) so if you don't want to have to load the tags on every template, add them to your global templatetags: http://www.djangosnippets.org/snippets/160/ http://www.djangosnippets.org/snippets/160/
- jsdalton 17y ago
  Not to get into a pedantic argument with you, friend, but the generally accepted definition of "model" in the MVC pattern is "the place that holds your business logic." See http://en.wikipedia.org/wiki/Model-view-controller http://en.wikipedia.org/wiki/Model-view-controller, for example (the article uses the term "domain logic," but it's the same thing - http://en.wikipedia.org/wiki/Domain_logic http://en.wikipedia.org/wiki/Domain_logic).
- phernandez 17y ago
megamark16 17y ago
I love django-evolution, I'm using it on AppRabbit.com to regenerate dynamic models and it's been a lifesaver. The only hitches I've found (granted the relationships between my models are fairly restrictive) are related to adding null=False fields and such. But really, it handles most things seamlessly.
goodness 17y ago
Man, a lot of those sound familiar. A lot of hard-coded path problems are especially annoying since the official Django documentation often tells you to do it that way. On a semi-related topic - I have an old 0.96 Django app that was using an old database migration framework that isn't supported in the newer versions of Django. For that framework I wrote the Django migrations in manually generated SQL. Anyone know how I could update to django-evolution or South with this hand coded SQL and still preserve my existing data? The documentation for django-evolution says: > Django Evolution imposes one important restriction on your development process: If you intend to make a change to your database, you must do it through Django Evolution. If you modify the database outside of Django and the Django Evolution framework, you're on your own. Which makes me think I can't use that.

126 points by niyazpk 17y ago

teye 17y ago

Great tips. When I was starting, some of these didn't occur to me and the others I knew I should be doing but didn't know how.

ivankirigin 17y ago

This is some really useful stuff. I learned a lot of this over time, and wished I had known it sooner. The point about schema evolution is important. I've taken to using a pickled python object in a text field in every model. That flexibility to store arbitrary data in a hash is amazing. I've used it to, for example, represent twitter accounts that are oauthed, with password, following the business account, or just nothing special. 4 kinds of accounts, with logic in the model to deal with it properly, all without evolving the schema. I only started with the first 2, and added the latter two without a problem. I'm pushing this a bit further now, and transitioning away from models altogether, and experimenting with a non-relational database like couchdb. I'm debating moving to google app engine too, but found some details (like getting on your own domain) to be pretty annoying.

seiji 17y ago
> I've taken to using a pickled python object in a text field in every model. That's basically what App Engine does right there. It stores by an id, then the value is your object serialized as a protocol buffer. The App Engine datastore allows you to look up by other properties than id because it indexes every field of your object in the background without you having to deal with it. You can query by pair-wise or more-wise indexes too, but you have to create those manually.
stanleydrew 17y ago
Storing pickled python objects is fine as long as you don't care about relational data. But if you aren't careful, data integrity will probably suffer over time as the complexity of your application increases. And that is not good. Anyway I'm sure a lot of people here know this stuff already. Just do your research and make sure the route you go makes sense for your application and your scaling requirements.
matclayton 17y ago
Really don't like the sound of that, pickling allows you to serialize any python code, and consequently deserialize it. Presumably you are placing user submitted data into this pickle and therefore without careful sanity checking wouldn't this potentially allow the user to inject code which may be run on deserialization? Someone else may be able to shed more light on this and correct me (please do!) but thats my concern and would love to hear if it was a valid one or not.
- limmeau 17y ago
  It is possible to build a poisonous pickle which, on depickling, calls arbitrary standard library functions, so accepting pickles from untrusted source is a No. One may think that if only the internal model of the user input is pickled, and the user only gets to update this internal model through web requests in defined ways, thus never getting to upload pickles or construct arbitrary complicated python objects, it should be relatively safe. However, what remains is that if someone manages to blindly inject SQL, he can inject executable code into your application, instead of just injecting data.
- patio11 17y ago
  Whoa! I was assuming this worked like doing a JSON grab-bag column in Rails or J2EE, which I am a fan of. Executable code would make me much, muuuuuuuuuch cooler on the idea. Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted?
  - RyanMcGreal 17y ago
    > Can you just do the same thing with JSON, Yaml, or your favorite serialization method of choice that does not have catastrophic consequences if parsing gets subverted? Sure. You can even extend the `json` module to serialize custom data types not defined in the JSON spec.
- ivankirigin 17y ago
  No, I'm just serializing dicts. Any user input is already in a string, so sanitized. I also usually make helper functions to get/set things in the dict, so I don't operate on the field outside the model often. This snippet is really helpful: http://www.djangosnippets.org/snippets/513/ http://www.djangosnippets.org/snippets/513/
FraaJad 17y ago
Pickling python objects and storing is an very old practice in Python world. Zope has been persisting objects in ZODB (a giant pickle jar with some transactional features) since 1998.
- ivankirigin 17y ago
  New to me :) I love it
banjiewen 17y ago
Would be interested to hear your take on Django+CouchDB - are you using a library like Couchdbkit, or rolling your own? Are you connecting it to the built-in admin?
- ivankirigin 17y ago
  Still exploring. I'll post something when it actually works.

hswolff 17y ago

Thank you for this. I am starting and hopefully this'll give me a solid foundation.

rmanocha 17y ago

Another point I would add to this list is to not be afraid of reading the Django Source code. While the docs cover a lot, you can find a lot of hidden gems simply going through the source code - it is highly readable and most stuff is where you would expect it to be. Of-course, be careful while doing so though 'cause the stuff that isn't documented can change at anytime, and may break your code.

xhuang 17y ago

thanks for this, i just started django development couple days ago, and this is very useful. another thing worth mention is that use generic view, it will save lots typing:)

wakeupthedawn 17y ago

These are pretty good. I really liked #6, the tip for template_tags. I hadn't thought of that before and will definitely do that. I will also look into extensions of the template system as suggested at the bottom. Those were really helpful, even for someone who has been using Django for a few months. I disagree with #4, the suggestion to not include business logic in views. What Django calls 'views' are what other frameworks call 'controllers' (MTV=MVC), and the business logic should reside there. I'm totally for separating the code into helper functions in other files, but putting business logic into models definitely violates the idea of MVC. For #5, I would recommend the second approach of having a separate settings.py for production vs development. Or at least code it where DEBUG=False is only overridden if socket confirms you are on a development machine - not the other way around. Debug contains too much info to risk exposing in production, but maybe my fear of socket screwing up is unfounded.

mhb 17y ago
For template tags, is doing it that way better than importing them in a base template which is included by the children templates?
- kennethlove 17y ago
  Templatetags don't inherit (at least not in my experience) so if you don't want to have to load the tags on every template, add them to your global templatetags: http://www.djangosnippets.org/snippets/160/ http://www.djangosnippets.org/snippets/160/
jsdalton 17y ago
Not to get into a pedantic argument with you, friend, but the generally accepted definition of "model" in the MVC pattern is "the place that holds your business logic." See http://en.wikipedia.org/wiki/Model-view-controller http://en.wikipedia.org/wiki/Model-view-controller, for example (the article uses the term "domain logic," but it's the same thing - http://en.wikipedia.org/wiki/Domain_logic http://en.wikipedia.org/wiki/Domain_logic).
phernandez 17y ago

megamark16 17y ago

I love django-evolution, I'm using it on AppRabbit.com to regenerate dynamic models and it's been a lifesaver. The only hitches I've found (granted the relationships between my models are fairly restrictive) are related to adding null=False fields and such. But really, it handles most things seamlessly.

goodness 17y ago

Man, a lot of those sound familiar. A lot of hard-coded path problems are especially annoying since the official Django documentation often tells you to do it that way. On a semi-related topic - I have an old 0.96 Django app that was using an old database migration framework that isn't supported in the newer versions of Django. For that framework I wrote the Django migrations in manually generated SQL. Anyone know how I could update to django-evolution or South with this hand coded SQL and still preserve my existing data? The documentation for django-evolution says: > Django Evolution imposes one important restriction on your development process: If you intend to make a change to your database, you must do it through Django Evolution. If you modify the database outside of Django and the Django Evolution framework, you're on your own. Which makes me think I can't use that.