16 ms·

Harmful Defaults in Django

55 points by jmconfuzeus 5y ago

BeFlatXIII 5y ago
An article about harmful defaults in Django with no mention of how the default settings.py encourages developers to commit secrets to their git history? Sad!
- jmconfuzeus 5y ago
  Good point! Django is full of surprises like this and it would take an eternity to enumerate all of them. inb4 some grumpy guy shoots me down for claiming that his favorite framework isn't perfect.
  - BeFlatXIII 5y ago
    I like Django: it’s admin interface is worth the price of admission, but encouraging people to put their database credentials directly into settings.py is setting developers up for failure. Anyone who claims “just be more diligent, bro” is someone who does not take security seriously.
- Nextgrid 5y ago
  I wish Django was more "12-factor" compliant by default. Reading secrets from environment variables should be a first-class citizen and it's not. I typically use python-decouple (https://github.com/henriquebastos/python-decouple/ https://github.com/henriquebastos/python-decouple/) for that but it would be great to have it as default.
aosaigh 5y ago
As someone has used Django extensively over the last 10 years for countless projects, I think that these are exaggerated claims of “harmful” defaults, particularly examples 2 to 6. Furthermore, this: > As you can see, Django doesn’t have your best interest at heart. Instead of providing sane defaults, it punishes you for not being a master at the framework. Is just nonsense. Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. It’s also heavily documented by Django and one of the first things mentioned. Lazy querysets are actually very useful when you need to build queries over a number of steps. Explicit query sets can have their own subtle issues with performance. 2. The default user model is fine for most users. It’s not a bad default and there is clear documentation on how to extend it. It’s designed to be extended! 3. I rarely rename migrations. Maybe this is a good idea, but it’s hardly a harmful default. 4. I almost never rename my tables. Again, maybe a sensible suggestion, not a harmful default. 5. Strangely this seems to be the most un-Django suggestion that would have the largest effect on a project but hardly any detail is given here. I never do this and wouldn’t recommend anyone else to when staring off. 6. I think project structure and CLI is one place where Django is weak but again this is hardly a harmful default. Use django-cookiecutter. Anyway I feel like the author is trying to manufacturer controversy to help define their expertise (not to say they aren’t a good Django developer, I have no idea). Some of these are good suggestions and “top tips” but they’re certainly not instances of Django trying to harm you.
- jmconfuzeus 5y ago
  Nothing I wrote is controversial. I explained my thought process at length but you flat out rejected everything without explaining why. I wonder who's creating controversy here.
- spookylukey 5y ago
  The default user model is fine, but I *always* start a project by extending it, for future-proofing. If you need to add to it, this is *so* much better than having to add a one-to-one, and at *some* point down the line you find you do want to add something. The problem is that doing it later when you need it is a pain, because of poor support by the migrations framework. It's like 4 lines of code that saves you so much grief later on: from django.contrib.auth.models import AbstractUser class User(AbstractUser): pass AUTH_USER_MODEL = "myapp.User" If you never need it, those 3 lines are not hurting you. In terms of a default, I think it probably would be nicer if Django pushed you to do this up front.
  - cessor 5y ago
    I agree 100%. Even if you don't need it, these lines don't hurt you, but most of the time, the additional model is put to good use. I usually add metadata, such as additional timestamps, flags, tags, custom user settings, consent or preferences.
- yxhuvud 5y ago
  > Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. No, there are frameworks out there that doesn't allow queries during template rendering. See Lucky for an example, https://luckyframework.org/ https://luckyframework.org/.
  - Nextgrid 5y ago
    The problem is not whether the queries are ran in templates or in code, it's which queries are ran. Doing the same he did in the template but in code (by constructing a dictionary of items to be passed to the template) will have exactly the same problem. The solution of using eager loading has nothing to do with whether you run your query in the template or in code.
samastur 5y ago
All of these "issues" are well documented if you bother to read just the main Django documentation. Some solutions are questionable. Executing querysets as soon as you define them means you can't compose them. Lazy quersets are not dangerous and you don't execute that many because they are lazy, but because under the hood you haven't asked for a JOIN statement. Automatic database table names are definitely not a problematic feature. 99% of the time it gives you very intuitive mapping between model name and database table name. If you actually want to move model to a different app, then you can use the same feature (db_table) to point to its old location. This article would benefit from more humility.
- jmconfuzeus 5y ago
  I wrote this post for beginners and most of them never read the docs before hacking on production code. Lazy querysets aren't dangerous but it's always best to evaluate them before rendering anything in order to avoid secret SQL queries. Many developers never run EXPLAIN, so these extra queries go unnoticed until your server crashes during Black Friday. I don't see why your database table needs to encapsulate the app's name. Most apps get rewritten at some point but databases can live for centuries. Why couple code with your data? Maybe I should be more humble to please sensitive people like you but no thanks. If a simple blog post hurts your feelings then you need to professional help.
  - Nextgrid 5y ago
    > I don't see why your database table needs to encapsulate the app's name. Because Django assumes that in most cases your DB will be primarily (only?) be used by the Django app. In my experience that is true of most startups and early-stage projects, and the default names have never been a blocker even when you start giving others access to that database (hopefully with a read-only user account). > Most apps get rewritten at some point but databases can live for centuries. You can rename tables when this becomes needed. I don't think it's worth wasting time doing so at the beginning (you could equally argue that whatever name you pick manually isn't going to be suitable in the future either, so a rename will be needed either way). > Why couple code with your data? Renaming tables doesn't realistically address that coupling by any means.
mherrmann 5y ago
As others here have pointed out, the article is much more clickbaity than its contents justify. None of the issues are dealbreakers and some expectations of the author are simply unreasonable. He complains that Django doesn't name generated files with a good human description of the intentions inside his head. I think we'll have to wait for Neuralink for that.
- cessor 5y ago
  Your're right, I felt the same way in that the article is clickbaity. I found that django's naming scheme for migrations is good enough; here are some real ones that I pulled out of my current project: migrations/0006_add_modeltranslation.py migrations/0007_remove_temp_fields.py migrations/0008_remove_language_field.py migrations/0009_alter_page_slug.py migrations/0010_alter_menuitem_absolute_url.py While they aren't semantically rich ('Why did you change absolute url on menu item?') I found that they are easy to distinguish and navigate.
  - jmconfuzeus 5y ago
    You pulled only the good ones. Where are the migrations named 0019_auto_20181224.py?
  - Klonoar 5y ago
    The only thing that annoys me to this day is that they have the numbers first. I get the logic why: it means they're ordered in the directory properly. That said, it's very annoying in that they're not tab-completable... any time I have to edit one, I have to stop and hunt in the entire list for what I'm looking for.
  - MattGaiser 5y ago
    "Why" is something very hard to programmically answer.
  - yxhuvud 5y ago
    My biggest objection to that naming scheme would be the numbering. If you have multiple people working on an app, it is likely that you at some point get multiples of a number as people add models simultaneously. What happens if there are multiples using a single number? I much prefer the timestamp based system that is used by Rails, it make the risk of that kind of conflict a lot lower.
samwillis 5y ago
I have been a user of Django for 15 years, I think I first used version 0.95 just after the “magic removal”. I wouldn’t say any of these are “harmful”, there are certainly some things here that have valid arguments for doing differently, but most is rubbish. But it needs to be taken into account that Django is an incredibly mature framework with literally millions of projects built on it. Large changes are hard, and Django has had to make many of them before. All of these have been debated and haven’t changed either due to immovable technical debt, a decision that there isn’t at this time a need, or there is valid arguments on both sides. A couple of examples: the user model, providing a default is important but it is also well documented that if you want to extend it you should subclass it from the beginning to make your life easier. There are ways to migrate later, they aren’t exactly easy, but it is doable. Multiple small apps, this architecture is due to the history of Django and its development at LWJ. They wanted a way of being able to quickly build new websites out of pre-defined “apps”, and this system works for that. At the time the Python packaging ecosystem was very different, hence then slightly odd design. I’m sure in hindsight the founding team would have done things differently, but this is 17 years ago. The general rule of thumb for “app” architecture now with Django is if you are a consultancy/agency building multiple similar sites for multiple clients (this is a huge use case for Django, closest to it original vision) it makes sense to package up your functionality into reusable apps. If you are a “startup” or building an internal app for a company with no need for reusability it’s best to just have one “core” app to contain all your functionality. It comes down to your personal needs, but it’s easy to make both architectures work.
bayesian_horse 5y ago
Django "apps" are sometimes reusable, but more often they aren't. For me the benefit in splitting a project into apps is that it provides a more manageable structure. You also avoid merge conflicts in teams, especially with migrations. I'm now working with Dotnet, because I have to (nobody wants to hire me as a Django developer even though I have 10+ years experience in Django and <0.5y in .NET), and this is an ongoing problem with EF core...
- bartonip 5y ago
  Oh mate we are hiring. Shoot me an email ;) hn@initialise.io
  - scandox 5y ago
    Lovely website - just a note that your contact pages / widgets all list support@example.com as your contact email...Ignore me if you're just building but thought it was worth pointing out.
- sgt 5y ago
  There's tons of Django jobs out there. Might just be under Python developers.
  - bayesian_horse 5y ago
    Believe me, I've tried. People commended my technical skills and still rejected me. Between a screwed up CV und unwillingness to move outside Germany during the pandemic, it just never worked out.
    - Nextgrid 5y ago
      Out of curiosity, why is your CV screwed up? Feel free to reach out (email in my profile) if you need advice and/or are looking for remote Django-focused work.
cessor 5y ago
The author names three 'harmful' defaults: 1. Relying on implicit SQL queries 2. Using the default User model 3. Using automatic migration names 4. Relying on automatic database table names 5. Using multiple tiny apps 6. Dumping apps in the root directory I wouldn't call those harmful. Most are non-issues, while others are "unfortunate" at best. 1. I found that the ORM works quite well and it's easy to optimize later. Using `prefetch_related` or `select_related` in the ORM still feels weird (they're just joins), but I wouldn't call them harmful. 2. I found it unfortunate that the info on how to change the User model is hidden so deep in the Documentation. I don't like to RTFM, but with Django it sure is worth it; same goes for 1. 3. I occasionally do it, but most of the time, I don't and never had an issue. 4. It do rely on that and found django's naming scheme to be a non-issue. I guess this might be more important if you need to change your database out of band (out of django band, that is) 5. This point is unsubstantiated. Apps aren't mere packages but also define a scope for the admin tool (which I use a lot and think is an integral part of Django) so that it will group models and allow individual access (i.e. apps also provide scope for your permissions). I use tons of apps but I wouldn't call it a harmful default. 6. Yeah, dynamic languages might produce naming collisions. Still beats imports like `org.example.publib.boulder.yourmoms.maidenname.models.impl.Account`
- jmconfuzeus 5y ago
  I consider them harmful for beginners who need to start working on non trivial production code. Multiple apps are definitely good for the admin but what I was advocating isn't 100% avoidance of apps but to avoid creating apps for every tiny feature. Personally, I like my databases to be designed for longevity and avoid couple code to my data. Code can die at any point in time but data is an asset that needs to live for as long as the company is in operation.
  - Nextgrid 5y ago
    > beginners who need to start working on non trivial production code. If you're a beginner dropped into a non-trivial production codebase your best bet is to follow existing conventions until you gain enough experience to be able to evaluate such advice and consider it against your projects' and business' requirements.
madrox 5y ago
I've been using Django for over a decade. I don't think most of these are a big deal and outright disagree with others. Happy to offer my own criticisms of django to add value to this comment thread. My real criticism of Django is that, at long last, I think it's starting to look dated. Templating hasn't been the preferred way of making web app UIs for some time. Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Setting up a project to do this is cumbersome. While there are third party solves, Django's greatness comes in part from how opinionated it is. I understand DSF may see adding more javascript as out of scope, but in keeping with the "batteries included" philosophy I hope they one day embrace something in contrib.
- Nextgrid 5y ago
  I wish Django had an equivalent to Rails' "unobtrusive Javascript": https://guides.rubyonrails.org/working_with_javascript_in_rails.html https://guides.rubyonrails.org/working_with_javascript_in_ra...
- cessor 5y ago
  I'd love to hear your criticism of django, issues you encountered and problems you have solved :) > My real criticism of Django is that, at long last, I think it's starting to look dated. Actually, that is something I found really great about django. I found that django is reliable over a long period of time. The js/node/react/express/angular apps I had developed in the past (~2012+) can't be built these days without redoing major components. I never had these problems with django. As for the responsiveness of the results, I found that a good understanding of django's performance helps. Using a good caching strategy (on models, views + nginx) yields results have almost instant reactions, even though there is the whole HTTP to HTML roundtrip. While they do have the noticable page-reload semantics, I found that these are way easier to build and sometimes yield very enjoyable experiences for my users. After all, I'm not building a YouTube so PictureInPicture and SPA-semantics aren't that important to my users. > Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Julia Elman explains how to do this with Django + DRF and Backbone.js in the book "Lightweight Django" (O'Reilly). Is this a viable compromise?
steve_mcdougall 5y ago
Wow this article is so poor it is almost malicious. > This means that you don’t own your database — the Django foundation does! This is a completely baseless accusation. At no point are you unable to change the schema or replace the user model (which we have done with admittedly some kerfuffle). If the author is reading this comment, and this is the reaction you were looking for from the community. Congratulations. You have written a completely asinine article about a technology you obviously do not understand yourself. Telling people to rename tables because "a future data analyst is not going to know what a table is for". On what planet is a data analyst given raw access to the production instance's tables of a database and not a view on a read replica. Having one migration per model is dumb as all hell, as you would need to manually separate these, achieving nothing of value in the process. Know-it-all anuses like the author of the article is why software engineering has become to be known as "a black hole of cash" because these are the type of people who pretend to know what they are talking about but make decisions that cost people exponential money and time. Please stop writing sensationalist, clickbait bullshit like this and please, please read at least a small part of the documentation before writing something misleading like this.
- jmconfuzeus 5y ago
  1. How are you supposed to change the concrete User model that Django provides without forking Django itself? It's 1000x better to subclass AbstractUser and provide your own User model that you own. 2. One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file. I wish you'd think straight instead of being angry. It's not like I'm claiming that the earth is flat.
  - Nextgrid 5y ago
    > One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file. Squashing is something Django will handle for you through a management command, at which point it doesn't matter whether you're doing one model per migration file or not. Stuff that cannot be squashed (RunSQL/Python operations) will be pointed out by Django and I'm not sure how having one model per migration will help - the difficulty here isn't locating the offending operation, it's to actually figure out whether that operation is still needed and if so whether it can be reworked.
drpancake 5y ago
I looked for a simple solution to this recently and I couldn't find one. It would be great if implicit SQL queries that occur as a result of missing out either select_related() or prefetch_related() would generate a console warning. I think this should be the default behaviour given that most beginning Django developers fall into this trap.
- jmconfuzeus 5y ago
  I agree. A warning should be the minimum because I don't see why anyone would consider hitting the database when accessing template context normal.
  - Nextgrid 5y ago
    The Django logging configuration is IMO unnecessarily complex and while it can be tuned to do anything with enough effort, I wish there was a one-click setting that says "tell me shit I should probably be aware of". I always find myself searching as to how to log SQL queries and copy/pasting the entire "LOGGING" setting temporarily despite working with the framework for several years.
nomilk 5y ago
I think these are simply "gotchas" rather than reasons not to use django. We've probably all committed an n+1 query at some point, but that doesn't mean the language/framework we were using was bad. Meta point: I really enjoy articles that list the places noobies go wrong (for any language/framework, not exclusively django). E.g. an experienced dev I taught R to was mind-blown that vec[1] accesses the first element of a vector (not vec[0]). Long lists of these unexpected (albeit simple) things can quickly be compiled. It might only take 30 minutes for a noobie to read but probably ends up saving days of pain down the track.
- jmconfuzeus 5y ago
  I never said that people need to stop using Django. It's still my favorite framework and will continue to be. I just wanted to point out some mistakes that many beginners make and that I made when I first started. Personally, I think Django could use better opinions from the core team because beginners should be set for success by default. In the current state, that's not the case.
  - nomilk 5y ago
    > I never said that people need to stop using Django. I know :) I was alluding to that exact point. I think other commenters thought that's what you meant, and jumped to the defence of the framework. But I agree that criticism is healthy, and it doesn't mean you don't enjoy or would stop using it. As someone who hasn't done much django, I appreciate your tips for beginners (especially in list format).
Ndymium 5y ago
I worked on Django projects for around 7–8 years professionally. Most of these weren't really issues, though I think we did run into name conflicts at some point with an app vs package. But the first pain point I very much agree with. Implicitly sending queries to the backend is one of those things that is an attempt to be very helpful to programmers, but that ends up as an obstacle that you always need to be wary of and avoid. Over time I have grown to appreciate explicitness in queries more and more, though I probably wouldn't go so far to remove lazy queries completely since they can be handy in query building, just remove the implicit loading feature if possible. Django's query builder is also quite far removed from SQL. Especially once you get into more complex queries, I sometimes just went raw SQL as I knew I wouldn't be able to remember how to translate the Django expression into SQL in a month's time. Simple queries are easy to write and look very sleek, but complex ones can turn into a mess that is much more understandable in SQL form. The sleekness of the queries may also betray how complex they are to execute as it's not "in your face". I still like Django's ORM quite a bit for many things, but after discovering Elixir's Ecto, I've enjoyed how explicit it is most of the time. I haven't needed implicit loading of relations once. It's also much closer to actual SQL. But there are things where Ecto also tries to be too helpful in my taste, like the humanised types in migrations (i.e. :utc_datetime instead of :timestamptz or :string instead of :"character varying"/:text) -- I strongly believe it would be easier in the long run for the programmer to specify the database types they want directly rather than through aliases which may result in surprising choices. Of course, that would cause issues in supporting multiple different databases, but that's not a goal I would go for in the first place, but I digress… What I guess I'm saying is that sometimes projects build in helpful features to make the code nicer to look at, more approachable to beginners, or just shorter to write, but they often end up being a burden on those that go past the beginner stage. People invariably trip on them because who can remember everything mentioned in all the documentation? And then it's just a game of "who fails to avoid the trap every time" and that's a game that everyone loses eventually.
- jmconfuzeus 5y ago
  Django's ORM is one of the best out there but you'd be surprised to know how many people don't know about things like prefetch_related. There are whole DjangoCon conference talks that go in length into these features even though I find them obvious. Makes you realize how many beginners the Django community actually has.
phn 5y ago
I mostly agree with other commenters, but I really must hard disagree with point 5. Proper abstraction is the one thing you need to get right, and django makes it so easy to make clean separate modules work together using a shared base and way of working (user model, groups, permissions, urls, signals, etc.). One day you'll need to use your payments logic to sell something else, embrace it.
- jmconfuzeus 5y ago
  If my payment logic is getting too coupled, I can easily extract it because I use one migration file per model and separate code using regular Python packages. I can easily extract the logic into it's own app if I need to publish it to PyPi for others to enjoy as well. Of course, if you code is getting coupled, then you need to revise your design process.
- hsbauauvhabzb 5y ago
  I wish the tutorial really explored this more. I’ve got some apps which have clear boundaries, but could be split into distinctive modules but I don’t see much value in doing so. I’d love to see an official example of this.
pmontra 5y ago
I relate with the author about what a Django's database could become if not handled with care. In my experience if one creates models using a lot of inheritance then the database becomes a dumpster and it's basically impossible to get any data out of it with something different than Python itself. The only sane way is to think about tables first, create models that mimic those tables, use makemigrations + migrate and absolutely no inheritance in the models. An extra bonus is only one app per project. I've got a customer with two projects: one with many apps and a lot of inheritance, the other one with one big app and a few very small ones and no inheritance. We can use any SQL tool to extract data from the database of the second project. The database of the first one is "spaghetti data" and "python3 manage.py shell" is the only way to make sense of it. On the other side we could say that at last a djangified database is a truly object oriented database.
- jmconfuzeus 5y ago
  You'd be surprised how many developers are working on Django projects right now who have never written a line of SQL in their entire lives. Instead of thinking about data and business logic, they're preoccupied by code and separation of concerns. That's why we end up with database that are unqueryable from outside of Django's ORM itself. Good luck if you end up working in a project that makes heavy use of GenericForeignKeys.
- Nextgrid 5y ago
  Model inheritance is indeed hell. Doesn't necessarily mean it should be removed (it might be a good shortcut and might be fine depending on your performance requirements) but it should be handled with care for sure. Same with generic foreign keys. I don't necessarily agree with your objections around how Django apps structure their database though. The need to access said database outside of Django should be a business requirement that's considered and if needed taken into account when designing the models. For projects that don't have said requirement it's not a problem because nothing meaningful should be accessing the DB (and if it does, then the application should be changed). It's kind of complaining that some third-party application's internal data file format doesn't suit your needs - well yes obviously it as it was never designed with you in mind in the first place.
sudmishra 5y ago
This is really a trash article. Don't even bother reading it.
- jmconfuzeus 5y ago
  I'm sorry you didn't find it useful. No need to dissuade others from reading it, especially beginners, because it will help them write better Django code.
  - jgb1984 5y ago
    Better? Half the article is opinion and taste, the other half is simply wrong, as others have described here in detail.
  - qot 5y ago
    I found it useful. I've been using Django casually in hobby projects for a few years now and your article just taught me about the N+1 query problem, so thanks!

55 points by jmconfuzeus 5y ago

An article about harmful defaults in Django with no mention of how the default settings.py encourages developers to commit secrets to their git history? Sad!

jmconfuzeus 5y ago
Good point! Django is full of surprises like this and it would take an eternity to enumerate all of them. inb4 some grumpy guy shoots me down for claiming that his favorite framework isn't perfect.
- BeFlatXIII 5y ago
  I like Django: it’s admin interface is worth the price of admission, but encouraging people to put their database credentials directly into settings.py is setting developers up for failure. Anyone who claims “just be more diligent, bro” is someone who does not take security seriously.
Nextgrid 5y ago
I wish Django was more "12-factor" compliant by default. Reading secrets from environment variables should be a first-class citizen and it's not. I typically use python-decouple (https://github.com/henriquebastos/python-decouple/ https://github.com/henriquebastos/python-decouple/) for that but it would be great to have it as default.

aosaigh 5y ago

As someone has used Django extensively over the last 10 years for countless projects, I think that these are exaggerated claims of “harmful” defaults, particularly examples 2 to 6. Furthermore, this: > As you can see, Django doesn’t have your best interest at heart. Instead of providing sane defaults, it punishes you for not being a master at the framework. Is just nonsense. Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. It’s also heavily documented by Django and one of the first things mentioned. Lazy querysets are actually very useful when you need to build queries over a number of steps. Explicit query sets can have their own subtle issues with performance. 2. The default user model is fine for most users. It’s not a bad default and there is clear documentation on how to extend it. It’s designed to be extended! 3. I rarely rename migrations. Maybe this is a good idea, but it’s hardly a harmful default. 4. I almost never rename my tables. Again, maybe a sensible suggestion, not a harmful default. 5. Strangely this seems to be the most un-Django suggestion that would have the largest effect on a project but hardly any detail is given here. I never do this and wouldn’t recommend anyone else to when staring off. 6. I think project structure and CLI is one place where Django is weak but again this is hardly a harmful default. Use django-cookiecutter. Anyway I feel like the author is trying to manufacturer controversy to help define their expertise (not to say they aren’t a good Django developer, I have no idea). Some of these are good suggestions and “top tips” but they’re certainly not instances of Django trying to harm you.

jmconfuzeus 5y ago
Nothing I wrote is controversial. I explained my thought process at length but you flat out rejected everything without explaining why. I wonder who's creating controversy here.
spookylukey 5y ago
The default user model is fine, but I *always* start a project by extending it, for future-proofing. If you need to add to it, this is *so* much better than having to add a one-to-one, and at *some* point down the line you find you do want to add something. The problem is that doing it later when you need it is a pain, because of poor support by the migrations framework. It's like 4 lines of code that saves you so much grief later on: from django.contrib.auth.models import AbstractUser class User(AbstractUser): pass AUTH_USER_MODEL = "myapp.User" If you never need it, those 3 lines are not hurting you. In terms of a default, I think it probably would be nicer if Django pushed you to do this up front.
- cessor 5y ago
  I agree 100%. Even if you don't need it, these lines don't hurt you, but most of the time, the additional model is put to good use. I usually add metadata, such as additional timestamps, flags, tags, custom user settings, consent or preferences.
yxhuvud 5y ago
> Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. No, there are frameworks out there that doesn't allow queries during template rendering. See Lucky for an example, https://luckyframework.org/ https://luckyframework.org/.
- Nextgrid 5y ago
  The problem is not whether the queries are ran in templates or in code, it's which queries are ran. Doing the same he did in the template but in code (by constructing a dictionary of items to be passed to the template) will have exactly the same problem. The solution of using eager loading has nothing to do with whether you run your query in the template or in code.

samastur 5y ago

All of these "issues" are well documented if you bother to read just the main Django documentation. Some solutions are questionable. Executing querysets as soon as you define them means you can't compose them. Lazy quersets are not dangerous and you don't execute that many because they are lazy, but because under the hood you haven't asked for a JOIN statement. Automatic database table names are definitely not a problematic feature. 99% of the time it gives you very intuitive mapping between model name and database table name. If you actually want to move model to a different app, then you can use the same feature (db_table) to point to its old location. This article would benefit from more humility.

jmconfuzeus 5y ago
I wrote this post for beginners and most of them never read the docs before hacking on production code. Lazy querysets aren't dangerous but it's always best to evaluate them before rendering anything in order to avoid secret SQL queries. Many developers never run EXPLAIN, so these extra queries go unnoticed until your server crashes during Black Friday. I don't see why your database table needs to encapsulate the app's name. Most apps get rewritten at some point but databases can live for centuries. Why couple code with your data? Maybe I should be more humble to please sensitive people like you but no thanks. If a simple blog post hurts your feelings then you need to professional help.
- Nextgrid 5y ago
  > I don't see why your database table needs to encapsulate the app's name. Because Django assumes that in most cases your DB will be primarily (only?) be used by the Django app. In my experience that is true of most startups and early-stage projects, and the default names have never been a blocker even when you start giving others access to that database (hopefully with a read-only user account). > Most apps get rewritten at some point but databases can live for centuries. You can rename tables when this becomes needed. I don't think it's worth wasting time doing so at the beginning (you could equally argue that whatever name you pick manually isn't going to be suitable in the future either, so a rename will be needed either way). > Why couple code with your data? Renaming tables doesn't realistically address that coupling by any means.

mherrmann 5y ago

As others here have pointed out, the article is much more clickbaity than its contents justify. None of the issues are dealbreakers and some expectations of the author are simply unreasonable. He complains that Django doesn't name generated files with a good human description of the intentions inside his head. I think we'll have to wait for Neuralink for that.

cessor 5y ago
Your're right, I felt the same way in that the article is clickbaity. I found that django's naming scheme for migrations is good enough; here are some real ones that I pulled out of my current project: migrations/0006_add_modeltranslation.py migrations/0007_remove_temp_fields.py migrations/0008_remove_language_field.py migrations/0009_alter_page_slug.py migrations/0010_alter_menuitem_absolute_url.py While they aren't semantically rich ('Why did you change absolute url on menu item?') I found that they are easy to distinguish and navigate.
- jmconfuzeus 5y ago
  You pulled only the good ones. Where are the migrations named 0019_auto_20181224.py?
- Klonoar 5y ago
  The only thing that annoys me to this day is that they have the numbers first. I get the logic why: it means they're ordered in the directory properly. That said, it's very annoying in that they're not tab-completable... any time I have to edit one, I have to stop and hunt in the entire list for what I'm looking for.
- MattGaiser 5y ago
  "Why" is something very hard to programmically answer.
- yxhuvud 5y ago
  My biggest objection to that naming scheme would be the numbering. If you have multiple people working on an app, it is likely that you at some point get multiples of a number as people add models simultaneously. What happens if there are multiples using a single number? I much prefer the timestamp based system that is used by Rails, it make the risk of that kind of conflict a lot lower.

samwillis 5y ago

I have been a user of Django for 15 years, I think I first used version 0.95 just after the “magic removal”. I wouldn’t say any of these are “harmful”, there are certainly some things here that have valid arguments for doing differently, but most is rubbish. But it needs to be taken into account that Django is an incredibly mature framework with literally millions of projects built on it. Large changes are hard, and Django has had to make many of them before. All of these have been debated and haven’t changed either due to immovable technical debt, a decision that there isn’t at this time a need, or there is valid arguments on both sides. A couple of examples: the user model, providing a default is important but it is also well documented that if you want to extend it you should subclass it from the beginning to make your life easier. There are ways to migrate later, they aren’t exactly easy, but it is doable. Multiple small apps, this architecture is due to the history of Django and its development at LWJ. They wanted a way of being able to quickly build new websites out of pre-defined “apps”, and this system works for that. At the time the Python packaging ecosystem was very different, hence then slightly odd design. I’m sure in hindsight the founding team would have done things differently, but this is 17 years ago. The general rule of thumb for “app” architecture now with Django is if you are a consultancy/agency building multiple similar sites for multiple clients (this is a huge use case for Django, closest to it original vision) it makes sense to package up your functionality into reusable apps. If you are a “startup” or building an internal app for a company with no need for reusability it’s best to just have one “core” app to contain all your functionality. It comes down to your personal needs, but it’s easy to make both architectures work.

bayesian_horse 5y ago

Django "apps" are sometimes reusable, but more often they aren't. For me the benefit in splitting a project into apps is that it provides a more manageable structure. You also avoid merge conflicts in teams, especially with migrations. I'm now working with Dotnet, because I have to (nobody wants to hire me as a Django developer even though I have 10+ years experience in Django and <0.5y in .NET), and this is an ongoing problem with EF core...

bartonip 5y ago
Oh mate we are hiring. Shoot me an email ;) hn@initialise.io
- scandox 5y ago
  Lovely website - just a note that your contact pages / widgets all list support@example.com as your contact email...Ignore me if you're just building but thought it was worth pointing out.
sgt 5y ago
There's tons of Django jobs out there. Might just be under Python developers.
- bayesian_horse 5y ago
  Believe me, I've tried. People commended my technical skills and still rejected me. Between a screwed up CV und unwillingness to move outside Germany during the pandemic, it just never worked out.
  - Nextgrid 5y ago
    Out of curiosity, why is your CV screwed up? Feel free to reach out (email in my profile) if you need advice and/or are looking for remote Django-focused work.

cessor 5y ago

The author names three 'harmful' defaults: 1. Relying on implicit SQL queries 2. Using the default User model 3. Using automatic migration names 4. Relying on automatic database table names 5. Using multiple tiny apps 6. Dumping apps in the root directory I wouldn't call those harmful. Most are non-issues, while others are "unfortunate" at best. 1. I found that the ORM works quite well and it's easy to optimize later. Using `prefetch_related` or `select_related` in the ORM still feels weird (they're just joins), but I wouldn't call them harmful. 2. I found it unfortunate that the info on how to change the User model is hidden so deep in the Documentation. I don't like to RTFM, but with Django it sure is worth it; same goes for 1. 3. I occasionally do it, but most of the time, I don't and never had an issue. 4. It do rely on that and found django's naming scheme to be a non-issue. I guess this might be more important if you need to change your database out of band (out of django band, that is) 5. This point is unsubstantiated. Apps aren't mere packages but also define a scope for the admin tool (which I use a lot and think is an integral part of Django) so that it will group models and allow individual access (i.e. apps also provide scope for your permissions). I use tons of apps but I wouldn't call it a harmful default. 6. Yeah, dynamic languages might produce naming collisions. Still beats imports like `org.example.publib.boulder.yourmoms.maidenname.models.impl.Account`

jmconfuzeus 5y ago
I consider them harmful for beginners who need to start working on non trivial production code. Multiple apps are definitely good for the admin but what I was advocating isn't 100% avoidance of apps but to avoid creating apps for every tiny feature. Personally, I like my databases to be designed for longevity and avoid couple code to my data. Code can die at any point in time but data is an asset that needs to live for as long as the company is in operation.
- Nextgrid 5y ago
  > beginners who need to start working on non trivial production code. If you're a beginner dropped into a non-trivial production codebase your best bet is to follow existing conventions until you gain enough experience to be able to evaluate such advice and consider it against your projects' and business' requirements.

madrox 5y ago

I've been using Django for over a decade. I don't think most of these are a big deal and outright disagree with others. Happy to offer my own criticisms of django to add value to this comment thread. My real criticism of Django is that, at long last, I think it's starting to look dated. Templating hasn't been the preferred way of making web app UIs for some time. Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Setting up a project to do this is cumbersome. While there are third party solves, Django's greatness comes in part from how opinionated it is. I understand DSF may see adding more javascript as out of scope, but in keeping with the "batteries included" philosophy I hope they one day embrace something in contrib.

Nextgrid 5y ago
I wish Django had an equivalent to Rails' "unobtrusive Javascript": https://guides.rubyonrails.org/working_with_javascript_in_rails.html https://guides.rubyonrails.org/working_with_javascript_in_ra...
cessor 5y ago
I'd love to hear your criticism of django, issues you encountered and problems you have solved :) > My real criticism of Django is that, at long last, I think it's starting to look dated. Actually, that is something I found really great about django. I found that django is reliable over a long period of time. The js/node/react/express/angular apps I had developed in the past (~2012+) can't be built these days without redoing major components. I never had these problems with django. As for the responsiveness of the results, I found that a good understanding of django's performance helps. Using a good caching strategy (on models, views + nginx) yields results have almost instant reactions, even though there is the whole HTTP to HTML roundtrip. While they do have the noticable page-reload semantics, I found that these are way easier to build and sometimes yield very enjoyable experiences for my users. After all, I'm not building a YouTube so PictureInPicture and SPA-semantics aren't that important to my users. > Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Julia Elman explains how to do this with Django + DRF and Backbone.js in the book "Lightweight Django" (O'Reilly). Is this a viable compromise?

steve_mcdougall 5y ago

Wow this article is so poor it is almost malicious. > This means that you don’t own your database — the Django foundation does! This is a completely baseless accusation. At no point are you unable to change the schema or replace the user model (which we have done with admittedly some kerfuffle). If the author is reading this comment, and this is the reaction you were looking for from the community. Congratulations. You have written a completely asinine article about a technology you obviously do not understand yourself. Telling people to rename tables because "a future data analyst is not going to know what a table is for". On what planet is a data analyst given raw access to the production instance's tables of a database and not a view on a read replica. Having one migration per model is dumb as all hell, as you would need to manually separate these, achieving nothing of value in the process. Know-it-all anuses like the author of the article is why software engineering has become to be known as "a black hole of cash" because these are the type of people who pretend to know what they are talking about but make decisions that cost people exponential money and time. Please stop writing sensationalist, clickbait bullshit like this and please, please read at least a small part of the documentation before writing something misleading like this.

jmconfuzeus 5y ago
1. How are you supposed to change the concrete User model that Django provides without forking Django itself? It's 1000x better to subclass AbstractUser and provide your own User model that you own. 2. One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file. I wish you'd think straight instead of being angry. It's not like I'm claiming that the earth is flat.
- Nextgrid 5y ago
  > One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file. Squashing is something Django will handle for you through a management command, at which point it doesn't matter whether you're doing one model per migration file or not. Stuff that cannot be squashed (RunSQL/Python operations) will be pointed out by Django and I'm not sure how having one model per migration will help - the difficulty here isn't locating the offending operation, it's to actually figure out whether that operation is still needed and if so whether it can be reworked.

drpancake 5y ago

I looked for a simple solution to this recently and I couldn't find one. It would be great if implicit SQL queries that occur as a result of missing out either select_related() or prefetch_related() would generate a console warning. I think this should be the default behaviour given that most beginning Django developers fall into this trap.

jmconfuzeus 5y ago
I agree. A warning should be the minimum because I don't see why anyone would consider hitting the database when accessing template context normal.
- Nextgrid 5y ago
  The Django logging configuration is IMO unnecessarily complex and while it can be tuned to do anything with enough effort, I wish there was a one-click setting that says "tell me shit I should probably be aware of". I always find myself searching as to how to log SQL queries and copy/pasting the entire "LOGGING" setting temporarily despite working with the framework for several years.

nomilk 5y ago

I think these are simply "gotchas" rather than reasons not to use django. We've probably all committed an n+1 query at some point, but that doesn't mean the language/framework we were using was bad. Meta point: I really enjoy articles that list the places noobies go wrong (for any language/framework, not exclusively django). E.g. an experienced dev I taught R to was mind-blown that vec[1] accesses the first element of a vector (not vec[0]). Long lists of these unexpected (albeit simple) things can quickly be compiled. It might only take 30 minutes for a noobie to read but probably ends up saving days of pain down the track.

jmconfuzeus 5y ago
I never said that people need to stop using Django. It's still my favorite framework and will continue to be. I just wanted to point out some mistakes that many beginners make and that I made when I first started. Personally, I think Django could use better opinions from the core team because beginners should be set for success by default. In the current state, that's not the case.
- nomilk 5y ago
  > I never said that people need to stop using Django. I know :) I was alluding to that exact point. I think other commenters thought that's what you meant, and jumped to the defence of the framework. But I agree that criticism is healthy, and it doesn't mean you don't enjoy or would stop using it. As someone who hasn't done much django, I appreciate your tips for beginners (especially in list format).

Ndymium 5y ago

I worked on Django projects for around 7–8 years professionally. Most of these weren't really issues, though I think we did run into name conflicts at some point with an app vs package. But the first pain point I very much agree with. Implicitly sending queries to the backend is one of those things that is an attempt to be very helpful to programmers, but that ends up as an obstacle that you always need to be wary of and avoid. Over time I have grown to appreciate explicitness in queries more and more, though I probably wouldn't go so far to remove lazy queries completely since they can be handy in query building, just remove the implicit loading feature if possible. Django's query builder is also quite far removed from SQL. Especially once you get into more complex queries, I sometimes just went raw SQL as I knew I wouldn't be able to remember how to translate the Django expression into SQL in a month's time. Simple queries are easy to write and look very sleek, but complex ones can turn into a mess that is much more understandable in SQL form. The sleekness of the queries may also betray how complex they are to execute as it's not "in your face". I still like Django's ORM quite a bit for many things, but after discovering Elixir's Ecto, I've enjoyed how explicit it is most of the time. I haven't needed implicit loading of relations once. It's also much closer to actual SQL. But there are things where Ecto also tries to be too helpful in my taste, like the humanised types in migrations (i.e. :utc_datetime instead of :timestamptz or :string instead of :"character varying"/:text) -- I strongly believe it would be easier in the long run for the programmer to specify the database types they want directly rather than through aliases which may result in surprising choices. Of course, that would cause issues in supporting multiple different databases, but that's not a goal I would go for in the first place, but I digress… What I guess I'm saying is that sometimes projects build in helpful features to make the code nicer to look at, more approachable to beginners, or just shorter to write, but they often end up being a burden on those that go past the beginner stage. People invariably trip on them because who can remember everything mentioned in all the documentation? And then it's just a game of "who fails to avoid the trap every time" and that's a game that everyone loses eventually.

jmconfuzeus 5y ago
Django's ORM is one of the best out there but you'd be surprised to know how many people don't know about things like prefetch_related. There are whole DjangoCon conference talks that go in length into these features even though I find them obvious. Makes you realize how many beginners the Django community actually has.

phn 5y ago

I mostly agree with other commenters, but I really must hard disagree with point 5. Proper abstraction is the one thing you need to get right, and django makes it so easy to make clean separate modules work together using a shared base and way of working (user model, groups, permissions, urls, signals, etc.). One day you'll need to use your payments logic to sell something else, embrace it.

jmconfuzeus 5y ago
If my payment logic is getting too coupled, I can easily extract it because I use one migration file per model and separate code using regular Python packages. I can easily extract the logic into it's own app if I need to publish it to PyPi for others to enjoy as well. Of course, if you code is getting coupled, then you need to revise your design process.
hsbauauvhabzb 5y ago
I wish the tutorial really explored this more. I’ve got some apps which have clear boundaries, but could be split into distinctive modules but I don’t see much value in doing so. I’d love to see an official example of this.

pmontra 5y ago

I relate with the author about what a Django's database could become if not handled with care. In my experience if one creates models using a lot of inheritance then the database becomes a dumpster and it's basically impossible to get any data out of it with something different than Python itself. The only sane way is to think about tables first, create models that mimic those tables, use makemigrations + migrate and absolutely no inheritance in the models. An extra bonus is only one app per project. I've got a customer with two projects: one with many apps and a lot of inheritance, the other one with one big app and a few very small ones and no inheritance. We can use any SQL tool to extract data from the database of the second project. The database of the first one is "spaghetti data" and "python3 manage.py shell" is the only way to make sense of it. On the other side we could say that at last a djangified database is a truly object oriented database.

jmconfuzeus 5y ago
You'd be surprised how many developers are working on Django projects right now who have never written a line of SQL in their entire lives. Instead of thinking about data and business logic, they're preoccupied by code and separation of concerns. That's why we end up with database that are unqueryable from outside of Django's ORM itself. Good luck if you end up working in a project that makes heavy use of GenericForeignKeys.
Nextgrid 5y ago
Model inheritance is indeed hell. Doesn't necessarily mean it should be removed (it might be a good shortcut and might be fine depending on your performance requirements) but it should be handled with care for sure. Same with generic foreign keys. I don't necessarily agree with your objections around how Django apps structure their database though. The need to access said database outside of Django should be a business requirement that's considered and if needed taken into account when designing the models. For projects that don't have said requirement it's not a problem because nothing meaningful should be accessing the DB (and if it does, then the application should be changed). It's kind of complaining that some third-party application's internal data file format doesn't suit your needs - well yes obviously it as it was never designed with you in mind in the first place.

sudmishra 5y ago

This is really a trash article. Don't even bother reading it.

jmconfuzeus 5y ago
I'm sorry you didn't find it useful. No need to dissuade others from reading it, especially beginners, because it will help them write better Django code.
- jgb1984 5y ago
  Better? Half the article is opinion and taste, the other half is simply wrong, as others have described here in detail.
- qot 5y ago
  I found it useful. I've been using Django casually in hobby projects for a few years now and your article just taught me about the N+1 query problem, so thanks!