7 ms·
I feel like Django has too much magic. I agree that it is easier to get quick wins with Django, and if you're building a small hobby site or something, then it
by jedberg 9y ago
I feel like Django has too much magic. I agree that it is easier to get quick wins with Django, and if you're building a small hobby site or something, then it's great.
But if you're building something that you hope you'll have to rapidly scale, Django is going to hurt you. It's way harder to scale due to both its heavy reliance on hidden magic its tight integration with its data store.
When you want to rapidly scale, the easiest way to do that is if your data store and application aren't so tightly intertwined that you have to scale both to solve a bottleneck in either.
- coldtea 9y ago>But if you're building something that you hope you'll have to rapidly scale, Django is going to hurt you. It's way harder to scale due to both its heavy reliance on hidden magic its tight integration with its data store. Considering that Django has 1/5th as much "magic" as Rails, and that Rails powers some of the biggest websites on the world, I'd say "citation needed".
- jrs95 9y agoAnd Django powers some huge sites as well. IMO the magic doesn't really hurt you there as long as your still working with a relatively monolithic piece of software.
- Steeeve 9y agoWhat besides reddit?
- 147 9y agoReddit isn't powered by Django as far as I know. But Instagram is and I believe Pinterest was/is.
- codefreakxff 9y agoNov 2016 https://medium.com/@Pinterest_Engineering/how-we-switched-our-template-rendering-engine-to-react-a799a3d540b0 https://medium.com/@Pinterest_Engineering/how-we-switched-ou...
- CCing 9y agoBackend is still django though
- sthatipamala 9y agoThe Pinterest web backend has some vestiges of Django for form validations and stuff, but IIRC it's mostly custom at this point.
- flunhat 9y agoInstagram is one, I'm sure there's others [1]. [1] https://engineering.instagram.com/web-service-efficiency-at-instagram-with-python-4976d078e366 https://engineering.instagram.com/web-service-efficiency-at-...
- Veratyr 9y agoReddit is Pylons based. Instagram is Django though.
- dom0 9y agoReddit uses Pylons iirc But see for yourself: https://github.com/reddit/reddit https://github.com/reddit/reddit
- Steeeve 9y agoInteresting. Looking around because I thought it was ruby at one point, but it turns out it was originally written in Lisp. http://www.aaronsw.com/weblog/rewritingreddit http://www.aaronsw.com/weblog/rewritingreddit I don't know where I got that idea from.
- agumonkey 9y agoAnd they blogged about moving to typescript for frontend at least
- wyc 9y agoKept a small list here, including codebase changes made for scaling and references: https://www.wyc.io/posts/just-build-it-in-django-or-rails/ https://www.wyc.io/posts/just-build-it-in-django-or-rails/ Instagram, Disqus, Bitbucket, Pinterest. Apologies in advanced for any marketing copies you don't want to read. The information is there.
- brynedwards 9y agoThere is a list at the bottom of this page: https://www.djangoproject.com/start/overview/ https://www.djangoproject.com/start/overview/ Disqus Instagram Knight Foundation MacArthur Foundation Mozilla National Geographic Open Knowledge Foundation Pinterest Open Stack
- yen223 9y agoIf the answer here is to be believed, Pinterest has consciously moved away from Django. https://www.quora.com/Would-Pinterest-consider-Flask-in-place-of-Django-if-it-were-starting-today https://www.quora.com/Would-Pinterest-consider-Flask-in-plac...
- coldtea 9y agoYes, but that doesn't matter much, since they got to billion valuations just fine WITH Django.
- mugsie 9y agoOnly one of the main Open Stack projects uses Django - and it is a complete pain to work on as a result. Most of the projects in OpenStack use Flask or Pecan - or Falcon for one or two of the more performance sensitive APIs
- gejjaxxita 9y agoHaving used Django and that project (Horizon) extensively, I think that Horizon is not a pain to work on as a result of Django, it was just badly written.
- Jugurtha 9y agoI believe python.org, too, is built on Django[0]. Also, here's an interesting thread on Artima: "Please Teach me Web Frameworks for Python!"[1] by Guido van van Rossum[sic]. It's 11 years old (a lot has changed since?), but interesting. [0]: https://github.com/python/pythondotorg https://github.com/python/pythondotorg [1]: http://www.artima.com/weblogs/viewpost.jsp?thread=146149 http://www.artima.com/weblogs/viewpost.jsp?thread=146149
- danso 9y agoPyCon's keynote this year was given by Instagram: https://www.youtube.com/watch?v=66XoCk79kjM https://www.youtube.com/watch?v=66XoCk79kjM
- a3n 9y agoWhat is "magic,' and how do you measure it so that you can say things like "1/5?"
- coldtea 9y agoEver seen Penn and Teller? Joking aside, generally we call "magic" uses of reflection, monkey-patching, auto-magically configured parts, parts of a framework that do way more than what we explicitly tell them, etc. "1/5 less magic" is what we call in casual conversation "making a point" and not an actual measurement. It is however more than 85% factually correct. The Django project leaders have stated that one of their goals was to not have "magic" in the framework, and in fact, one of the early (~ Django 1.2 or so) refactoring efforts was about removing "magic" too-clever parts for more explicitness.
- zimablue 9y agoWhich parts are you referring to as magic, the models? I use DRF and find the way it deals with defaults quite nice, your can use a model and it will generate fields or not, and specify or generate them yourself.
- jedberg 9y agoMostly the ORM. If you rely on it, you can't really lay out your tables in any way except the way they do it, at least not without a whole lot of headache.
- njharman 9y agoIt requires you have a primary key. That's it. Everything else table names, fk's, multiple databases, etc. is overridable. The only thing I know of that is a slight headache is postgresql namespaces (aka schemas). Django does make choices for you. But almost everyone of those choices can be overridden. I have found that most people who think Django (or any large/mature/widely used project) can't to something is because they don't know it well enough.
- tedmiston 9y agoNot sure what you mean by this. You have full control over your table schemas and relationships with the Django ORM.
- dom0 9y agoDjango's magic and some really "Unpythonic" implementations of magic are mostly annoying when you're trying to use it in any context other than serving exactly one webapp over HTTP per process. (Unrelated blurb:) Pyramid's design contains a lot of wisdom; some folks working on it have been doing Python web frameworks since the web came into existence.
- tuxxy 9y agoI completely agree. My favorite framework is Flask, but I have to work with Django professionally. I find that using Flask produces much cleaner, readable code (depending on the engineer, of course). When I'm working with Django code, I find myself opening two to four different files just to make a tiny change in some restful API. I find myself doing the same just trying to read some code or trace down a bug or even a utility function. I find that the object oriented nature of Django also tends to produce overly complex codebases for the things that people try to create. It hides code in favor of magic which just complicates the development process.
- ris 9y ago> I feel like Django has too much magic Are you kidding? Flask is side-effecty module, global-variables, make-sure-you-put-this-magic-incantation-which-you-dont-understand-in-your-init everywhere.
- jedberg 9y agoI never said Flask was better...
- Veratyr 9y agoAs someone who's worked on a moderately large Django project, I have to disagree. There's a lot of stuff done for you without you seeing it (like Admin and the default QuerySets + Managers) but it's fairly easy to understand, customize or debug if you want to and Django's source code is actually really readable. Plus, Django has some of the best software documentation I've ever seen. It was incredibly rare that I couldn't find an answer to how/why something worked in the docs.
- orf 9y agoDjango has a lot less magic than Flask. Flask has thread-local global request objects for example, among other things. Under the hood Django is quite explicit in how it works. What examples of magic do you have, specifically?
- Walkman 9y agoDjango has hidden thread locals, Flask makes them explicit, so a lot less magic.
- orf 9y agoI was interested so I had a look at the source. There is a single use of a threading.local object in the RegexResolver. There are 5 other usages in total, one in the live reloader, two in the tests and two in GeoDjango. Explicit magic is still magic, and Django contains surprisingly little magic of any type overall, including thread locals.
- Walkman 9y agowhat about the ORM? https://github.com/django/django/blob/master/django/db/utils.py https://github.com/django/django/blob/master/django/db/utils...
- orf 9y agoGood point, I was only searching for explicit "threading.local" references. So there are 4 more or so. Django uses thread locals in some places where it's needed, but that's beside the point though. It doesn't put thread local objects at the core of its API. Passing an explicit request (real or fake) into functions is much more testable and generally better than relying on a 'request' global, IMO. Same with all the other app globals [1]. A better, more specific argument about Django's use of magic is the models. It uses some dark black magic voodo metaclasses to implement those. Apart from that, the forms, admin, app registry, urls, views is pretty straightforward. But hey, it's personal preference. I like flask mostly: for some situations it is amazing. 1. http://flask.pocoo.org/docs/0.12/api/#application-globals http://flask.pocoo.org/docs/0.12/api/#application-globals
- tony 9y agoDjango does have magic: The key thing is Django's implicit behavior is directly proportional to the job it's trying to do [1]: - Django's settings can be invoked at any time. So it's kept as a lazily-loaded singleton that isn't processed until an attribute is first accessed. - Django needs to know which apps to load (a la INSTALLED_APPS). And to know the models, django needs to know the apps. These app models can contain relationship dependencies, so its necessarily from a invocation perspective to have a settings that declare them all. - Using Django's initialization is a must. For the reasons above (settings), but there is work done behind the scenes to make sure everything lines up when Django starts. - It's most likely project's using Django are going to be using it's ORM. On the other hand: - Django doesn't hold every project, or every request for that matter, to its batteries. Intricacies like middleware and context processors are opt-in. Things like template filters are also opt-in on a per-file basis via {% load %} tags. - Whether or not to use Forms Framework / crispy-forms, for instance, is optional. But projects would miss out on nicely generated forms with validation. - Projects can also switch out template engines for Jinja2 or something totally custom. - While strictly speaking, projects can just keep models.py empty, QuerySet's are like the ultimate reusable interface in Django's Frameworks. QuerySet's can be used: - by class-based views (CBV) - ModelForms to provide database-backed form validation - by third party extensions like Django REST Framework (DRF), django-filter, django-tables2, django-guardian, and so on - in context processors (which can access URL regex group matches), that later are passed into in templates. While that may feel like overkill, this is useful for things like database-backed menus that CMS like Drupal and WordPress would use by default. > It's way harder to scale due to both its heavy reliance on hidden magic its tight integration with its data store. I plan on making a future article diving into Django's ORM. Sure, ORM's hide a lot of machinery behind a facade of sugary objects. But on most web projects the database's aren't doing much more than simple joins. Django's ORM can scale into medium-sized code bases. By the time it gets beyond the point the ORM can handle, heavier data tools (like ElasticSearch, Hadoop) would end up being brought in regardless of whether Django was picked. That doesn't mean earlier code or data schemas need to be thrown out. Put it another way: When project's get to the point they need a datamart or other things, they're likely to create a separate service for that distinctive from the web front-end. [1] https://www.git-pull.com/code_explorer/django-vs-flask.html#configuring-django https://www.git-pull.com/code_explorer/django-vs-flask.html#... / https://www.git-pull.com/code_explorer/django-vs-flask.html#django-s-intialization https://www.git-pull.com/code_explorer/django-vs-flask.html#...
- ceyhunkazel 9y agoScalability problem is the nicest thing that can happen to you which means you are successful. Short time to market and low budget are the common cases for startups. If you have money and time then you can more worry about scalability issues and all generalizations are false, including this one.
- sametmax 9y agoNa, I built a video streaming website with django. It has 400 000 v/d. The team now working on it is not very gifted, and yet they manage to make it evolve because Django makes it easy.
- askvictor 9y agoI also thought it was magic and scary when I started using it. Then I got to understand how it all works and it's no longer magic, just python.
- nine_k 9y agoDjango uses "magic" (mostly in the form of "convention over configuration") to make certain approaches simpler. It also makes other approaches, not planned for by the framework, harder. This is why Django is great for a particular kind of sites, and not as great for other kinds. "Originally designed for the newsroom", as they note on their site; it still feels. As it's typical with frameworks, Django helps you as long as you do things "the Django way", and does not help, or counteracts you, if your problem domain does not fit well into Django's. I suspect it's the case with any reasonably specific tool. A hammer is damn efficient as long as you have to deal with nails, and a wrench is great for nuts and bolts — but neither is going to help you to assemble an electronic device. Choosing the right tool for the job is as important as always. Django, on the other hand, is not a tool, but a toolset ("framework"). Replacing any tool in it is not impossible but loses some of the affordances of the original custom-adjusted tool. I used SQLAlchemy with Django, and it definitely can be done, but you lose e.g. automatic admin pages. I had to go to immense lengths to change the way settings initialization works in Django when I had to update some of them after `import settings`. Flask is a more loosely coupled bunch of tools. E.g. SQLAlchemy is an entirely separate project, just adopted to work in Flask with a custom layer. From the point of view of long-term development, it's better, it's more flexible. From the POV of a rapid prorotyping of a typical CRUD app, Django is better because it's more integrated. It will bite later, but at the start, most people are ready to make technical debt in exchange for velocity.
- RUG3Y 9y agoI'm currently working on rebuilding a fairly large Django app in Flask. My situation is exactly what you describe, the use case is not one which Django is really built to accomplish. The result is lots of difficulty dealing with the data. I feel like Django is great if you're building content driven websites. Our website barely scratches the surface of our application, and the Django backend has become very.
- fatbird 9y agoAbility to "rapidly scale" is an overrated virtue. Scaling is exactly when you should step back and reflect on your architecture, at which point you should understand that very few of your scaling concerns are at the framework level. I continue to be involved with a 2+ million LOC Django site that's been 8 years under continuous development that went from 10+ million users to 100+ million a summer ago. Django was never the issue. If the particularities of Django are what's in the way of scaling for you, you have bigger issues than Django.
- mychael 9y ago> If you're building something that you hope you'll have to rapidly scale, Django is going to hurt you. Yet somehow people have built wildly successful companies handling massive amounts of traffic using Django. Yes, many have had to re-architect to scale better, but which company hasn't had to do that as they grow from thousands to millions of daily active users?