17 ms·

Django 3

657 points by cstuder 7y ago

cstuder 7y ago
Release announcement: https://www.djangoproject.com/weblog/2019/dec/02/django-3-released/ https://www.djangoproject.com/weblog/2019/dec/02/django-3-re... (The release note erroneously still show the "Under Development" banner.)
pkuphy 7y ago
https://www.aeracode.org/2018/06/04/django-async-roadmap/ https://www.aeracode.org/2018/06/04/django-async-roadmap/ https://github.com/django/deps/blob/master/accepted/0009-async.rst https://github.com/django/deps/blob/master/accepted/0009-asy...
IgorPartola 7y ago
Groovy. I am excited to try out the new ASGI goodness. Last time I used Django Channels it was a constant struggle to tune it, prevent it from becoming deadlocked, etc. And the transition from Channels 1.x to 2.x was also not pleasant. I hope that Django proper did things better/differently.
- haspok 7y ago
  Not Groovy, Python! :)
  - brento 7y ago
    Adding a smiley doesn't negate that your comment was unnecessary and of no added value to the conversation.
    - shhshahassa 7y ago
      It's a joke you psycho
    - Eldt 7y ago
      Being negative doesn't negate that your comment was unnecessary and of no added value to the conversation.
    - mataniko 7y ago
      It a reference to the Groovy language - http://groovy-lang.org/ http://groovy-lang.org/
- onepointsixC 7y ago
  Do you know if this update changes how Django Channels works?
- emptysea 7y ago
  I really wanted to use Django channels for some soft real-time stuff like chat and notifications and it worked but I couldn’t get testing to work reliably (tests would just hang sometimes), so I had to use a different setup.
- andrewgodwin 7y ago
  Well there's no actual async support for views in 3.0 - we missed the cutoff due to performance concerns! Hopefully it'll be in 3.1.
  - naveen_ 7y ago
    Thanks for the info & for your work!
bicx 7y ago
Does Python still have performance issues? That's always been my concern when considering its adoption.
- pytester 7y ago
  It's always prioritized delivery speed and quality over raw performance. If you want an ultra optimized web framework it will always be the wrong choice - it was never designed for that.
- heinrichhartman 7y ago
  nothing significant changed.
- ageitgey 7y ago
  The are some new-ish features like async and there some new parallelization options in 3.8 that help in some very specific cases, but generally you choose Python because you value development speed more than execution speed. You can use Python (or Ruby, etc) to build fast websites that scale well if you are smart about how you use it and how you design your application, but it still isn't a particularly fast language compared to others.
- sojmq 7y ago
  Yes. If you can afford the CPU power, by all means go with Python. Otherwise just don't. Instagram can afford it so it works fine for them. Reddit can't, so their site is always slow or down.
  - traverseda 7y ago
    I think reddits issues are more related to database IO than cpu time.
  - kashug 7y ago
    And you know it is becouse of python reddit is often slow or down? Or are you just blaming pythin without any actual data on the underlying reasons?
  - 7y ago
scarygliders 7y ago
Right as I was just about to start a new Django project... The thought going through my mind right now is "I wonder if there is some way of creating a Django project and make it resilient to future Django updates and releases with minimum fuss?" I've had to deal with ongoing and inherited legacy projects which run on Python 2.7, use the long-deprecated Pylons, for which there is no easy upgrade path other than a total re-write in less deprecated technologies. No practical way to convert that Pylons project to Pyramid, for example. I guess the answer to my thought above is to "Not do/use anything /too exciting/ when creating your Django project" - i.e. only use the most basic and generic Django features, but even then what's to say those won't get deprecated in favour of some new exciting replacement? I guess that's one of the dilemmas developers have day to day?
- ealexhudson 7y ago
  Well, there were a few things in the 2.x series (e.g. naming of default auth views) which ended up breaking existing code and needed updates. But in all honesty: django is the project I would worry about the least here. Even when stuff is deprecated, an improved alternative tends to exist and is pretty well documented. The level of care that goes into each release is very high; the number of brown bag bugs that get out very low. If you want a quiet life, django is a very good fit.
- danpalmer 7y ago
  > i.e. only use the most basic and generic Django features I don't think you need to do this with Django. Features are generally very mature, stable, and are not deprecated frequently. When they are, they are deprecated years in advance with long term support still provided. In all honesty, I've seen very few projects run feature lifecycles as well as Django. I'd use everything you need to in it without concern about this.
  - StavrosK 7y ago
    Seconded, I haven't really had many problems upgrading Django projects, and I've upgraded a lot of them.
darkerside 7y ago
For those on the 1.11 LTS release, and considering getting off the LTS train, I wonder whether it's a feasible upgrade path to go straight to 3.0, or if it would be smarter to go to 2.2 first.
- onepointsixC 7y ago
  I suspect it would be far kinder to yourself to first upgrade from 1.11 to 2.2 and get all your tests working and then consider upgrading to 3.0.
- jacobian 7y ago
  My experience has been that the easiest method is to to upgrade one minor release at once (e.g. in your case I'd do 1.11 -> 2.0 -> 2.1 -> 2.2 -> 3.0). This has almost always gone quite smoothly, but it is somewhat tedious. That said, jumping a bunch of releases (e.g. 1.11 -> 3.0) actually often works out just fine, especially if I've got good test coverage. It's just that when it doesn't work, it can be tricky to find out what's going wrong. That's because the release notes are typically written with just a single step in mind, so with a bunch of releases I find myself scrambling to figure out which set of notes I need to read.
  - samwillis 7y ago
    I recently went from 1.11 -> 2.0 -> 2.1 -> 2.2 over the course of about a week, deploying to production each time we had all test passing to ensure there wasn't anything amiss. There were actually relatively few changes to get there, the upgrade to Python 3 was a bigger (albeit still manageable) change. I'm expecting very few changes to get onto Django 3.0, will probably wait till the first bug fix release though as it tends to catch a few things. The best thing to go is run the dev server with `python -Wd` to get the depreciation warnings, and fix them before each upgrade. Worked perfectly!
- stefano 7y ago
  I recently upgraded from 1.11 to 2.2 in one go. I didn't have any issue at all, besides a problem with mysql drivers. I had to switch from using PyMySQL, which is not the one recommended by Django anyway, to mysqlclient, and that caused one minor problem that was trivial to fix. Looking at the release notes for 3.0, I'd expect that if I migrated from 1.11 directly to 3.0 it would've been a smooth transition. Of course it depends on your codebase. In the past I worked on a project which interacted a lot with Django internals, with a lot of code reflection involved. Upgrading that was a pain even between minor releases.
Pandabob 7y ago
This looks neat. Does anyone have experience with using type hints and mypy with Django? Any gotchas?
- truebosko 7y ago
  No specific gotchas with type hinting, but be aware that Django has not implemented any notion of type hints. So, you lack quite the benefit of type hinting
- henbruas 7y ago
  I'm currently experimenting with it, so I don't have a lot to say yet. But you'll want to use this: https://pypi.org/project/django-stubs/ https://pypi.org/project/django-stubs/
- emptysea 7y ago
  I gave ‘django-stubs’ a spin on a Django 2.2 project. The ORM typings and mypy plug-in worked great. ‘.filter().first()’ and similar all returned the correct types. Ran into a couple issues with the typing of ‘client.force_login()’ where the ‘user’ parameter was typed as the ‘get_django_user()’ which wasn’t the same as the custom user model I defined in the ‘models.py’. I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’. You might be able to override the few types that don’t work for your given setup or you can just fork the repo. Overall, I think the pain is worth the benefits.
  - mkurnikov 7y ago
    Hi, I'm a primary maintainer of 'django-stubs'. We're going to release new version with mypy==0.750 and Django 3.0 support soon. Could you create an issue for your 'client.force_login' problem? > I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’. This one also has plugin support, and should be working mostly correct. Fill the issue if you can reproduce, I'd love to help with that. What's your experience with that whole "plugin has to actually run django.setup() before typecheck" thing?
- 7y ago
randshift 7y ago
Congrats to Django's contributors and community! It's been a while since I've actively worked in Django but I found the people in the Django ecosystem to be generally very helpful.
Ramiuz 7y ago
I have been using FastAPI for the last two months (which also is an ASGI server and makes full use of annotations and type hints with mypy) and the experience has been incredible. (https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/) If Django can now also support annotations and async code I dream of an scenario where apis can be built using this two elements. Does anybody know a good resource to learn/catch up with this release?
- gonational 7y ago
  I recommend the release notes for each version, starting with the one right after the last version you’ve used. There’s a couple reasons: 1. you get a breakdown of all the new features, which only takes a few minutes to kind of quickly go through each version and decide which bits you care about 2. you get a list of backwards-incompatible changes, which resolves any upgrade regressions, as long as you’re not using internal APIs that have changed
- drieddust 7y ago
  How hard was the learning curve? What are the main benefits over Django?
  - m_ke 7y ago
    Learning curve is super low. Main benefits are: 1. Performance, it's a wrapper on top of starlette/uvicorn, which brings the performance closer to nodejs (https://www.techempower.com/benchmarks/#section=data-r17&hw=ph&test=fortune&l=zijzen-1 https://www.techempower.com/benchmarks/#section=data-r17&hw=...). (I did run into some issues with it though due to the default response validation when serializing large response bodies) 2. Lightweight background tasks (from starlette) 3. Documentation generation from type annotations. It's a nice tool for microservices but coming from django you'll have to roll your own database management, authentication, sessions, caching, admin and etc. I'm also not a fan of the magic request unpacking using type annotations and prefer getting a request object as is done in django and starlette. IMHO most people would probably be better off with plain starlette and a 3 line decorator to handle request validation and response serialization.
gonational 7y ago
If you can convince your company to keep Django up-to-date with every release, I have found this to be easy and pain-free (Django has been pretty API stable since ~1.8), compared with waiting for each LTS and then being forced to jump ahead three versions each time. You also gain access to new features as they come out, this way.
- primozk 7y ago
  But you have to be careful that all dependencies are up-to-date as well.
  - vdfs 7y ago
    No to mention each new release have many incompatibility that you need to read the release note to find out
    - BozeWolf 7y ago
      Django is very good at being backwards compatible. Deprecated functions emit warnings for a few releases before being removed. They are also documented. Do you have some examples? Are you sure you are not confused with dependencies? The release notes mention less than ten deprecations related to django. Other stuff which is not supported anymore is mostly non supported database versions. https://docs.djangoproject.com/en/3.0/releases/3.0/#backwards-incompatible-3-0 https://docs.djangoproject.com/en/3.0/releases/3.0/#backward...
    - marcosdumay 7y ago
      The Django incompatibilities is a small problem. If you do sign-up for the theory that it's better to use its ecosystem, the changes on the related projects are much larger.
m_ke 7y ago
Was really excited to upgrade but the async safety check makes the ORM unusable in a Jupyter Notebook. https://stackoverflow.com/questions/59119396/how-to-use-django-3-0-orm-in-a-jupyter-notebook-without-triggering-the-async-con https://stackoverflow.com/questions/59119396/how-to-use-djan... https://forum.djangoproject.com/t/is-there-a-way-to-disable-the-synchronousonlyoperation-check-when-using-the-orm-in-a-jupyter-notebook/548 https://forum.djangoproject.com/t/is-there-a-way-to-disable-...
- singularity2001 7y ago
  please tell me that python did not adopt the javascript async hell. async should be an optional keyword on the CALLER side, not an invisible trait of the function. go figure()
  - m_ke 7y ago
    It didn't, async await is just syntactic sugar on top of coroutines/futures. You can define an sync function and get a coroutine back if you call it without await. It's still kind of a mess though because you have two universes that are not really compatible, forcing you to rewrite everything that touches IO. For django that probably means a rewrite of the ORM, caching and middlewares.
    - hajile 7y ago
      JS async/await is also just syntactic sugar on top of Promises (which are basically just futures with the ability to write as well).
  - jakear 7y ago
    In Javascript, async is an optional parameter on the caller side. (Well, 'await' is, but I figure that's what you mean). It can be used with any function call, not just those that are marked async. Though it only make sense to use it with Promise-returning functions. Alternatively, you can also use it with a plain Promise, which makes for a nice mutex. I've also used this for imitating async constructors, which is not currently possible.
deleted 7y ago
[deleted]
fareesh 7y ago
Interested in reading about agencies that used to (mostly) build web applications in Rails but then switched to (mostly) Django. I find the productivity somewhat higher from a business point of view, to turn around projects for customers when using Rails instead - largely due to the gem ecosystem and the way community libraries tend to play well with one another.
- m_ke 7y ago
  I think it really comes down to the type of project that you're working on. For data science / machine learning projects python is a much better choice and that spills over to django. REST APIs are also incredibly easy to throw together using DRF.
  - fareesh 7y ago
    I feel like you could just unload the data science + ML stuff to a separate process though.
ak39 7y ago
OT: Are there any backend JSON/REST open source applications* that allow declarative configuration for your storage (eg. set SQL statements in config files) instead of needing compilation?
deleted 7y ago
[deleted]
904baf11 7y ago
> ASGI support > Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application. Bleh. ASGI is probably the worst implementation I can think of for a common async interface. Stringly-typed callbacks? async generators exist for a good reason. It's also very asyncio-centric, and asyncio is Not Good.
- Ralfp 7y ago
  Can you explain why asyncio is not good?
  - 904baf11 7y ago
    It's convuluted, it's not structured concurrency (https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...), fundamentally despite being async/await a lot of networking code is based on callbacks anyway (see: asyncio.Protocol), and I've had a lot of trouble debugging code that spawn tasks because asyncio code has zero obligation to care about any of the tasks it spawns.
    - tomchristie 7y ago
      Yo. Two points here. * You can absolutely stay within structured concurrency constraints using ASGI. * The ASGI spec doesn’t have anything to do with asyncio. It’s purely an async/await interface - You can build against it just fine using other concurrency backends.
- andrewgodwin 7y ago
  I'd love to hear your suggestions for changes we could make while keeping it somewhat WSGI-compatible. It took a few years to refine it to where it is now, so it's not like we just threw something at the wall.
  - 904baf11 7y ago
    > while keeping it somewhat WSGI-compatible There's the problem. WSGI is fundamentally flawed too - it could also be using generators for a two-way communication channel instead of stringly typed callbacks. In a world where Python has optional static type hints, it would be nice to have concrete objects passed too.
reustle 7y ago
I've been building some fun side projects with Django lately and it's really a breath of fresh air. Normal, old, boring web applications with server side rendered html and basic forms. Sure, I can't add some super fast interactivity on my forms as easily without some json endpoints, but for most of my projects it's totally fine.
goodoldneon 7y ago
> Django is now aware of asynchronous event loops and will block you calling code marked as “async unsafe” - such as ORM operations - from an asynchronous context. So DB calls aren't async?
- gonational 7y ago
  Correct - the Django ORM is not (yet) async. The asyncpg library[1][2] could be used for direct, asynchronous database access, from within a Django 3.x app. The asyncpg library is a low-level async Postgres adaptor, similar to psycopg2, except that it provides much better abstractions and automatic encoding / decoding (vs just piping bytes back and forth like psycopg2), not to mention it is much faster than anything else out there. IOW, asyncpg would serve as a much better foundation for a future ORM. My best guess is that, in the future, Django's ORM will be refactored to optionally support async and will supply an adaptor for asyncpg. 1. https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg 2. https://magicstack.github.io/asyncpg/current/ https://magicstack.github.io/asyncpg/current/
  - fjp 7y ago
    aiopg is also great and has full SQLAlchemy support
  - hultner 7y ago
    Asyncpg is amazing, I’m writing a thin dataclass-style semi-orm-layer ontop is asyncpg. Still very early but I’m considering cleaning it up and extracting it to a stand-alone library if I see any demand for it.
tracer4201 7y ago
The last time I used django was for one piece of a capstone related project back in college, and for what I was doing, it was dead simple (to get up and running, that is). I’ve seen python used extensively for data science work, but I’m not sure how well dynamic typing works (in terms of support ability/maintenance) in a larger code base, especially as some trivial knowledge gets lost and newer folks are introduced to an older code base. Do you just set break points to trace? I digress, but open to hearing folks thoughts.
tabbott 7y ago
Zulip has been powered by Django since the very early days of its development with Django 1.4, back in 2012. As a reasonably mature web application with significant scale, we're at the stage in many companies' development where one starts to rip out more and more of the web framework to optimize things or just make them work the way we want. (E.g. while I was at Dropbox in early 2016, we discovered we only had about 600 lines of code left from the original Pylons framework that actually ran). One of the things that has been really fantastic about Django is that we're still happily using it for the vast majority of code in the project, and every time Django comes out with a new release, I read the changelog and get excited about several improvements that actually make my life better. Overall I think we've gotten a ton of value out of Python and Django and would recommend it to anyone starting a new full-featured web application project today. My only frustration with this release announcement is dropping Python 3.5 support now. Because Python 3.5 is not EOL and used in LTS OS releases that have vendor support through 2021, this forces us to choose between our policy of supporting vendor OS releases until they reach EOL, not upgrading to Django 3 for the next year or more, or shipping our own Python on Ubuntu Xenial.
- NickBusey 7y ago
  Thanks for your work on Zulip, I tend to be logged into at least 3 instances at any given time. Love the treaded model.
- georgyo 7y ago
  Zulip is quite amazing and I really like the product, however I somewhat dislike that the installer is a complete machine take over with many hard-coded paths. Supporting only Ubuntu releases is one thing, but the code base actively fights installing it on anything else but the supported distributions (ubuntu). At which point, why even have a installer, just ship an image. Since Zulip has so many moving components, I would love to see it be installed with something like Nix. Doing so would allow it to be installed in a consistent way across many different OSes, and decouple your reliance on software provided by the OS. It would also make upgrades seamless. You can install nix on any single linux distribution, not just nixos, and since your installer already hard requires running as root, you can install nix and then install zulip in nix.
luckycharms810 7y ago
Having worked with Asynchronous python since the days of twisted, tornado - I haven't really found any real reason to switch to asyncio. Generally speaking the old forms of async syntax (@coroutine, @inline_callback) make it incredibly clear that you are executing code well outside of a sequential paradigm - they force you to think a little more about what is going on. The idea of Django throwing errors when you try and run async unsafe code within an event loop gives me cause for concern. I can just imagine folks who don't understand adding 'yield', 'await', 'yield from' until the errors go away - while being confused with the resulting execution. I think there is always a place for synchronous code in Python - I think Django should lean in to the simplicity afforded by it. Last note - I also am very weary to mix asynchronous code with other primitives like threading, multiprocessing. If I am going the route of async programming - I usually just intend to optimize a single thread - and then I leave it up to the OS to optimize across multiple processes.
- j88439h84 7y ago
  > Having worked with Asynchronous python since the days of twisted, tornado - I haven't really found any real reason to switch to asyncio. Agree. On the other hand, Trio (https://trio.readthedocs.io/ https://trio.readthedocs.io/) gives me plenty of reasons to switch.
- fjp 7y ago
  I'm a really big fan of using aiohttp + aiopg + some light wrapping for CRUD APIs
- kissgyorgy 7y ago
  > I also am very weary to mix asynchronous code with other primitives like threading, multiprocessing. If I am going the route of async programming - I usually just intend to optimize a single thread - and then I leave it up to the OS to optimize across multiple processes. A lot of the time you simply can't not have another thread or process, for example, when you have a blocking library call or CPU intensive task.
wiremine 7y ago
Just want to say thank you to the Django core team. I was privileged to use Django full time for over 6 years. Both the project and the community are a joy. There's a few times I had to dive into Django's internals, and every time it was logical and straight forward to understand and modify. Thanks guys!
airstrike 7y ago
The community built around Django, coupled with the sheer amount of documentation and the stability of their APIs has done enough to forever ruin my expectations for any other coding project out there. Ostensibly everything else feels disorganized, poorly documented or rushly released. But the most commendable value of the DSF – which I believe is a model to be replicated – is how effectively it has managed to both attract new developers and also mentor them so well that they have gone on to become core developers who maintain the framework later on. I'm still subscribed to the django-developers mailing list, even if I have had no free time to contribute with more than a couple patches over many years, simply because I enjoy watching how incredibly smart and efficient developers collaborate. Kudos to all of you guys, and congrats on achieving this big milestone!
- madrox 7y ago
  I echo this sentiment exactly. I don't feel like OSS projects achieve this kind of success by accident. I'd love to see some write-up of some kind on how they do it. They're definitely the gold standard in my mind.
  - frankwiles 7y ago
    That’s a damn good idea!
deusofnull 7y ago
i <3 django. thank you to all the people involved!
Alir3z4 7y ago
Django and Python have made me a much better developer. Using it for more than 10 years professionally, full time with no stop. It helped me to pay my bills, bring food to the table and build my ideas and get one step closer to my dreams. Thank you Django and everyone around or behind or even remotely related to it.
- andy1729 7y ago
  Can you explain lil bit more like what exactly you do with django (freelancing or job??) and your journey with django. I am sensing their is a great story which can inspire newbies like me! Thank you very much sir!
  - Alir3z4 7y ago
    Prior to working with Django, I worked with ASP.NET/C# and later PHP with various web framework CakePHP, Zend, Symfony) and then worked Ruby on Rails for 2 years until I found Django (version 1.0.x I think) and from there in less than week, without even having a knowledge in Python, just by following tutorials and reading the source code I become productive and made production-ready apps. Almost all of my work with Django is through full time and once a while contracting gig here and there (very short ones.) Not just my day time job, I also implement all my side/personal/commercial projects with Django (gonevis.com is my latest project). Django never gave me a tough time to understand anything, for almost everything there's a well-maintained library and properly tested with enough community around it, it's ORM is crazy easy to work with, optimize, tweak and change when needed. Every new release is easy to upgrade, you may have some problem with third party libraries to catch up if something is not compatible with them but in my experience with medium-size code base (~300k LOC) the whole upgrade would take 2-3 hours and that was just changing code and running the tests, all in all, in 2-3 week most of the other third parties would have upgraded as well. It's mature framework, doesn't get crazy with new shiny things (NoSQL, MongoDB, WebSocket, etc), it might be late to some technology but it's because it stays in the corner until all these shiny things have worked out their problem and issues, then those things will become part of the official code base, otherwise if you're in hurry, there's always another third party that bring those for you, either NoSQL, WebSocket, push notifications, OTP, etc. For me, the most fascinating thing in Django is the ORM, Authentication, Admin, Views and template engine and the crazy support of PostgreSQL (almost the same applies to other database backends in Django), sessions, caching and every other single thing in that carefully have been layered upon each other and work in harmony. The database migration (formerly South migration then part of official) is something you can't easily find in other frameworks (Rails migration is great as well). Now, DRF is something else, since starting to work with DRF, I've become an API monster :D The framework that lets you implement your idea without getting in your way and allows you to bring ideas from zero to production very quickly, is the one I look forward to using (I use Java Spring once a while as well). For me, Django and it's surrounding makes sense, it's logical and it's not magical, maybe it's just me and it may not be something tasteful for others, if you find a framework you're comfortable with it, then use it, if you get into Django, I hope you feel the same.
andrewstuart 7y ago
I learned, in this order: bottle flask falcon django Django is incredible. It was useful to start with the minimalist frameworks to understand the true value of Django when I got there. Django is amongst my software top 10. Maybe top 5.
- oehtXRwMkIs 7y ago
  Curious as to what your top 10 is.
bobjordan 7y ago
Is substituting the Django ORM for SQLAlchemy doable in V3?
rptr_87 7y ago
I have been learning flask for development. Can someone describe advantages of using Django over flask?
- ablekh 7y ago
  Please check out the following posts: 1) https://devel.tech/features/django-vs-flask; https://devel.tech/features/django-vs-flask; 2) https://www.airpair.com/python/posts/django-flask-pyramid https://www.airpair.com/python/posts/django-flask-pyramid. Hope this helps.
aitchnyu 7y ago
OT, but what do you use to validate JSON posts without DRF? We mangled django form fields to validate a list of ids, but can I use forms to validate nested JSON? I now use Marshmallow for that use case but would love to stick with Django forms.
Hsoon 7y ago
Https://www.google.com

657 points by cstuder 7y ago

Release announcement: https://www.djangoproject.com/weblog/2019/dec/02/django-3-released/ https://www.djangoproject.com/weblog/2019/dec/02/django-3-re... (The release note erroneously still show the "Under Development" banner.)

pkuphy 7y ago

https://www.aeracode.org/2018/06/04/django-async-roadmap/ https://www.aeracode.org/2018/06/04/django-async-roadmap/ https://github.com/django/deps/blob/master/accepted/0009-async.rst https://github.com/django/deps/blob/master/accepted/0009-asy...

IgorPartola 7y ago

Groovy. I am excited to try out the new ASGI goodness. Last time I used Django Channels it was a constant struggle to tune it, prevent it from becoming deadlocked, etc. And the transition from Channels 1.x to 2.x was also not pleasant. I hope that Django proper did things better/differently.

haspok 7y ago
Not Groovy, Python! :)
- brento 7y ago
  Adding a smiley doesn't negate that your comment was unnecessary and of no added value to the conversation.
  - shhshahassa 7y ago
    It's a joke you psycho
  - Eldt 7y ago
    Being negative doesn't negate that your comment was unnecessary and of no added value to the conversation.
  - mataniko 7y ago
    It a reference to the Groovy language - http://groovy-lang.org/ http://groovy-lang.org/
onepointsixC 7y ago
Do you know if this update changes how Django Channels works?
emptysea 7y ago
I really wanted to use Django channels for some soft real-time stuff like chat and notifications and it worked but I couldn’t get testing to work reliably (tests would just hang sometimes), so I had to use a different setup.
andrewgodwin 7y ago
Well there's no actual async support for views in 3.0 - we missed the cutoff due to performance concerns! Hopefully it'll be in 3.1.
- naveen_ 7y ago
  Thanks for the info & for your work!

bicx 7y ago

Does Python still have performance issues? That's always been my concern when considering its adoption.

pytester 7y ago
It's always prioritized delivery speed and quality over raw performance. If you want an ultra optimized web framework it will always be the wrong choice - it was never designed for that.
heinrichhartman 7y ago
nothing significant changed.
ageitgey 7y ago
The are some new-ish features like async and there some new parallelization options in 3.8 that help in some very specific cases, but generally you choose Python because you value development speed more than execution speed. You can use Python (or Ruby, etc) to build fast websites that scale well if you are smart about how you use it and how you design your application, but it still isn't a particularly fast language compared to others.
sojmq 7y ago
Yes. If you can afford the CPU power, by all means go with Python. Otherwise just don't. Instagram can afford it so it works fine for them. Reddit can't, so their site is always slow or down.
- traverseda 7y ago
  I think reddits issues are more related to database IO than cpu time.
- kashug 7y ago
  And you know it is becouse of python reddit is often slow or down? Or are you just blaming pythin without any actual data on the underlying reasons?
- 7y ago

scarygliders 7y ago

Right as I was just about to start a new Django project... The thought going through my mind right now is "I wonder if there is some way of creating a Django project and make it resilient to future Django updates and releases with minimum fuss?" I've had to deal with ongoing and inherited legacy projects which run on Python 2.7, use the long-deprecated Pylons, for which there is no easy upgrade path other than a total re-write in less deprecated technologies. No practical way to convert that Pylons project to Pyramid, for example. I guess the answer to my thought above is to "Not do/use anything /too exciting/ when creating your Django project" - i.e. only use the most basic and generic Django features, but even then what's to say those won't get deprecated in favour of some new exciting replacement? I guess that's one of the dilemmas developers have day to day?

ealexhudson 7y ago
Well, there were a few things in the 2.x series (e.g. naming of default auth views) which ended up breaking existing code and needed updates. But in all honesty: django is the project I would worry about the least here. Even when stuff is deprecated, an improved alternative tends to exist and is pretty well documented. The level of care that goes into each release is very high; the number of brown bag bugs that get out very low. If you want a quiet life, django is a very good fit.
danpalmer 7y ago
> i.e. only use the most basic and generic Django features I don't think you need to do this with Django. Features are generally very mature, stable, and are not deprecated frequently. When they are, they are deprecated years in advance with long term support still provided. In all honesty, I've seen very few projects run feature lifecycles as well as Django. I'd use everything you need to in it without concern about this.
- StavrosK 7y ago
  Seconded, I haven't really had many problems upgrading Django projects, and I've upgraded a lot of them.

darkerside 7y ago

For those on the 1.11 LTS release, and considering getting off the LTS train, I wonder whether it's a feasible upgrade path to go straight to 3.0, or if it would be smarter to go to 2.2 first.

onepointsixC 7y ago
I suspect it would be far kinder to yourself to first upgrade from 1.11 to 2.2 and get all your tests working and then consider upgrading to 3.0.
jacobian 7y ago
My experience has been that the easiest method is to to upgrade one minor release at once (e.g. in your case I'd do 1.11 -> 2.0 -> 2.1 -> 2.2 -> 3.0). This has almost always gone quite smoothly, but it is somewhat tedious. That said, jumping a bunch of releases (e.g. 1.11 -> 3.0) actually often works out just fine, especially if I've got good test coverage. It's just that when it doesn't work, it can be tricky to find out what's going wrong. That's because the release notes are typically written with just a single step in mind, so with a bunch of releases I find myself scrambling to figure out which set of notes I need to read.
- samwillis 7y ago
  I recently went from 1.11 -> 2.0 -> 2.1 -> 2.2 over the course of about a week, deploying to production each time we had all test passing to ensure there wasn't anything amiss. There were actually relatively few changes to get there, the upgrade to Python 3 was a bigger (albeit still manageable) change. I'm expecting very few changes to get onto Django 3.0, will probably wait till the first bug fix release though as it tends to catch a few things. The best thing to go is run the dev server with `python -Wd` to get the depreciation warnings, and fix them before each upgrade. Worked perfectly!
stefano 7y ago
I recently upgraded from 1.11 to 2.2 in one go. I didn't have any issue at all, besides a problem with mysql drivers. I had to switch from using PyMySQL, which is not the one recommended by Django anyway, to mysqlclient, and that caused one minor problem that was trivial to fix. Looking at the release notes for 3.0, I'd expect that if I migrated from 1.11 directly to 3.0 it would've been a smooth transition. Of course it depends on your codebase. In the past I worked on a project which interacted a lot with Django internals, with a lot of code reflection involved. Upgrading that was a pain even between minor releases.

Pandabob 7y ago

This looks neat. Does anyone have experience with using type hints and mypy with Django? Any gotchas?

truebosko 7y ago
No specific gotchas with type hinting, but be aware that Django has not implemented any notion of type hints. So, you lack quite the benefit of type hinting
henbruas 7y ago
I'm currently experimenting with it, so I don't have a lot to say yet. But you'll want to use this: https://pypi.org/project/django-stubs/ https://pypi.org/project/django-stubs/
emptysea 7y ago
I gave ‘django-stubs’ a spin on a Django 2.2 project. The ORM typings and mypy plug-in worked great. ‘.filter().first()’ and similar all returned the correct types. Ran into a couple issues with the typing of ‘client.force_login()’ where the ‘user’ parameter was typed as the ‘get_django_user()’ which wasn’t the same as the custom user model I defined in the ‘models.py’. I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’. You might be able to override the few types that don’t work for your given setup or you can just fork the repo. Overall, I think the pain is worth the benefits.
- mkurnikov 7y ago
  Hi, I'm a primary maintainer of 'django-stubs'. We're going to release new version with mypy==0.750 and Django 3.0 support soon. Could you create an issue for your 'client.force_login' problem? > I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’. This one also has plugin support, and should be working mostly correct. Fill the issue if you can reproduce, I'd love to help with that. What's your experience with that whole "plugin has to actually run django.setup() before typecheck" thing?
7y ago

randshift 7y ago

Congrats to Django's contributors and community! It's been a while since I've actively worked in Django but I found the people in the Django ecosystem to be generally very helpful.

Ramiuz 7y ago

I have been using FastAPI for the last two months (which also is an ASGI server and makes full use of annotations and type hints with mypy) and the experience has been incredible. (https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/) If Django can now also support annotations and async code I dream of an scenario where apis can be built using this two elements. Does anybody know a good resource to learn/catch up with this release?

gonational 7y ago
I recommend the release notes for each version, starting with the one right after the last version you’ve used. There’s a couple reasons: 1. you get a breakdown of all the new features, which only takes a few minutes to kind of quickly go through each version and decide which bits you care about 2. you get a list of backwards-incompatible changes, which resolves any upgrade regressions, as long as you’re not using internal APIs that have changed
drieddust 7y ago
How hard was the learning curve? What are the main benefits over Django?
- m_ke 7y ago
  Learning curve is super low. Main benefits are: 1. Performance, it's a wrapper on top of starlette/uvicorn, which brings the performance closer to nodejs (https://www.techempower.com/benchmarks/#section=data-r17&hw=ph&test=fortune&l=zijzen-1 https://www.techempower.com/benchmarks/#section=data-r17&hw=...). (I did run into some issues with it though due to the default response validation when serializing large response bodies) 2. Lightweight background tasks (from starlette) 3. Documentation generation from type annotations. It's a nice tool for microservices but coming from django you'll have to roll your own database management, authentication, sessions, caching, admin and etc. I'm also not a fan of the magic request unpacking using type annotations and prefer getting a request object as is done in django and starlette. IMHO most people would probably be better off with plain starlette and a 3 line decorator to handle request validation and response serialization.

gonational 7y ago

If you can convince your company to keep Django up-to-date with every release, I have found this to be easy and pain-free (Django has been pretty API stable since ~1.8), compared with waiting for each LTS and then being forced to jump ahead three versions each time. You also gain access to new features as they come out, this way.

primozk 7y ago
But you have to be careful that all dependencies are up-to-date as well.
- vdfs 7y ago
  No to mention each new release have many incompatibility that you need to read the release note to find out
  - BozeWolf 7y ago
    Django is very good at being backwards compatible. Deprecated functions emit warnings for a few releases before being removed. They are also documented. Do you have some examples? Are you sure you are not confused with dependencies? The release notes mention less than ten deprecations related to django. Other stuff which is not supported anymore is mostly non supported database versions. https://docs.djangoproject.com/en/3.0/releases/3.0/#backwards-incompatible-3-0 https://docs.djangoproject.com/en/3.0/releases/3.0/#backward...
  - marcosdumay 7y ago
    The Django incompatibilities is a small problem. If you do sign-up for the theory that it's better to use its ecosystem, the changes on the related projects are much larger.

m_ke 7y ago

Was really excited to upgrade but the async safety check makes the ORM unusable in a Jupyter Notebook. https://stackoverflow.com/questions/59119396/how-to-use-django-3-0-orm-in-a-jupyter-notebook-without-triggering-the-async-con https://stackoverflow.com/questions/59119396/how-to-use-djan... https://forum.djangoproject.com/t/is-there-a-way-to-disable-the-synchronousonlyoperation-check-when-using-the-orm-in-a-jupyter-notebook/548 https://forum.djangoproject.com/t/is-there-a-way-to-disable-...

singularity2001 7y ago
please tell me that python did not adopt the javascript async hell. async should be an optional keyword on the CALLER side, not an invisible trait of the function. go figure()
- m_ke 7y ago
  It didn't, async await is just syntactic sugar on top of coroutines/futures. You can define an sync function and get a coroutine back if you call it without await. It's still kind of a mess though because you have two universes that are not really compatible, forcing you to rewrite everything that touches IO. For django that probably means a rewrite of the ORM, caching and middlewares.
  - hajile 7y ago
    JS async/await is also just syntactic sugar on top of Promises (which are basically just futures with the ability to write as well).
- jakear 7y ago
  In Javascript, async is an optional parameter on the caller side. (Well, 'await' is, but I figure that's what you mean). It can be used with any function call, not just those that are marked async. Though it only make sense to use it with Promise-returning functions. Alternatively, you can also use it with a plain Promise, which makes for a nice mutex. I've also used this for imitating async constructors, which is not currently possible.

deleted 7y ago

[deleted]

fareesh 7y ago

Interested in reading about agencies that used to (mostly) build web applications in Rails but then switched to (mostly) Django. I find the productivity somewhat higher from a business point of view, to turn around projects for customers when using Rails instead - largely due to the gem ecosystem and the way community libraries tend to play well with one another.

m_ke 7y ago
I think it really comes down to the type of project that you're working on. For data science / machine learning projects python is a much better choice and that spills over to django. REST APIs are also incredibly easy to throw together using DRF.
- fareesh 7y ago
  I feel like you could just unload the data science + ML stuff to a separate process though.

ak39 7y ago

OT: Are there any backend JSON/REST open source applications* that allow declarative configuration for your storage (eg. set SQL statements in config files) instead of needing compilation?

deleted 7y ago

[deleted]

904baf11 7y ago

> ASGI support > Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application. Bleh. ASGI is probably the worst implementation I can think of for a common async interface. Stringly-typed callbacks? async generators exist for a good reason. It's also very asyncio-centric, and asyncio is Not Good.

Ralfp 7y ago
Can you explain why asyncio is not good?
- 904baf11 7y ago
  It's convuluted, it's not structured concurrency (https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ https://vorpus.org/blog/notes-on-structured-concurrency-or-g...), fundamentally despite being async/await a lot of networking code is based on callbacks anyway (see: asyncio.Protocol), and I've had a lot of trouble debugging code that spawn tasks because asyncio code has zero obligation to care about any of the tasks it spawns.
  - tomchristie 7y ago
    Yo. Two points here. * You can absolutely stay within structured concurrency constraints using ASGI. * The ASGI spec doesn’t have anything to do with asyncio. It’s purely an async/await interface - You can build against it just fine using other concurrency backends.
andrewgodwin 7y ago
I'd love to hear your suggestions for changes we could make while keeping it somewhat WSGI-compatible. It took a few years to refine it to where it is now, so it's not like we just threw something at the wall.
- 904baf11 7y ago
  > while keeping it somewhat WSGI-compatible There's the problem. WSGI is fundamentally flawed too - it could also be using generators for a two-way communication channel instead of stringly typed callbacks. In a world where Python has optional static type hints, it would be nice to have concrete objects passed too.

reustle 7y ago

I've been building some fun side projects with Django lately and it's really a breath of fresh air. Normal, old, boring web applications with server side rendered html and basic forms. Sure, I can't add some super fast interactivity on my forms as easily without some json endpoints, but for most of my projects it's totally fine.

goodoldneon 7y ago

> Django is now aware of asynchronous event loops and will block you calling code marked as “async unsafe” - such as ORM operations - from an asynchronous context. So DB calls aren't async?

gonational 7y ago
Correct - the Django ORM is not (yet) async. The asyncpg library[1][2] could be used for direct, asynchronous database access, from within a Django 3.x app. The asyncpg library is a low-level async Postgres adaptor, similar to psycopg2, except that it provides much better abstractions and automatic encoding / decoding (vs just piping bytes back and forth like psycopg2), not to mention it is much faster than anything else out there. IOW, asyncpg would serve as a much better foundation for a future ORM. My best guess is that, in the future, Django's ORM will be refactored to optionally support async and will supply an adaptor for asyncpg. 1. https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg 2. https://magicstack.github.io/asyncpg/current/ https://magicstack.github.io/asyncpg/current/
- fjp 7y ago
  aiopg is also great and has full SQLAlchemy support
- hultner 7y ago
  Asyncpg is amazing, I’m writing a thin dataclass-style semi-orm-layer ontop is asyncpg. Still very early but I’m considering cleaning it up and extracting it to a stand-alone library if I see any demand for it.

tracer4201 7y ago

The last time I used django was for one piece of a capstone related project back in college, and for what I was doing, it was dead simple (to get up and running, that is). I’ve seen python used extensively for data science work, but I’m not sure how well dynamic typing works (in terms of support ability/maintenance) in a larger code base, especially as some trivial knowledge gets lost and newer folks are introduced to an older code base. Do you just set break points to trace? I digress, but open to hearing folks thoughts.

tabbott 7y ago

Zulip has been powered by Django since the very early days of its development with Django 1.4, back in 2012. As a reasonably mature web application with significant scale, we're at the stage in many companies' development where one starts to rip out more and more of the web framework to optimize things or just make them work the way we want. (E.g. while I was at Dropbox in early 2016, we discovered we only had about 600 lines of code left from the original Pylons framework that actually ran). One of the things that has been really fantastic about Django is that we're still happily using it for the vast majority of code in the project, and every time Django comes out with a new release, I read the changelog and get excited about several improvements that actually make my life better. Overall I think we've gotten a ton of value out of Python and Django and would recommend it to anyone starting a new full-featured web application project today. My only frustration with this release announcement is dropping Python 3.5 support now. Because Python 3.5 is not EOL and used in LTS OS releases that have vendor support through 2021, this forces us to choose between our policy of supporting vendor OS releases until they reach EOL, not upgrading to Django 3 for the next year or more, or shipping our own Python on Ubuntu Xenial.

NickBusey 7y ago
Thanks for your work on Zulip, I tend to be logged into at least 3 instances at any given time. Love the treaded model.
georgyo 7y ago
Zulip is quite amazing and I really like the product, however I somewhat dislike that the installer is a complete machine take over with many hard-coded paths. Supporting only Ubuntu releases is one thing, but the code base actively fights installing it on anything else but the supported distributions (ubuntu). At which point, why even have a installer, just ship an image. Since Zulip has so many moving components, I would love to see it be installed with something like Nix. Doing so would allow it to be installed in a consistent way across many different OSes, and decouple your reliance on software provided by the OS. It would also make upgrades seamless. You can install nix on any single linux distribution, not just nixos, and since your installer already hard requires running as root, you can install nix and then install zulip in nix.

luckycharms810 7y ago

Having worked with Asynchronous python since the days of twisted, tornado - I haven't really found any real reason to switch to asyncio. Generally speaking the old forms of async syntax (@coroutine, @inline_callback) make it incredibly clear that you are executing code well outside of a sequential paradigm - they force you to think a little more about what is going on. The idea of Django throwing errors when you try and run async unsafe code within an event loop gives me cause for concern. I can just imagine folks who don't understand adding 'yield', 'await', 'yield from' until the errors go away - while being confused with the resulting execution. I think there is always a place for synchronous code in Python - I think Django should lean in to the simplicity afforded by it. Last note - I also am very weary to mix asynchronous code with other primitives like threading, multiprocessing. If I am going the route of async programming - I usually just intend to optimize a single thread - and then I leave it up to the OS to optimize across multiple processes.

j88439h84 7y ago
> Having worked with Asynchronous python since the days of twisted, tornado - I haven't really found any real reason to switch to asyncio. Agree. On the other hand, Trio (https://trio.readthedocs.io/ https://trio.readthedocs.io/) gives me plenty of reasons to switch.
fjp 7y ago
I'm a really big fan of using aiohttp + aiopg + some light wrapping for CRUD APIs
kissgyorgy 7y ago
> I also am very weary to mix asynchronous code with other primitives like threading, multiprocessing. If I am going the route of async programming - I usually just intend to optimize a single thread - and then I leave it up to the OS to optimize across multiple processes. A lot of the time you simply can't not have another thread or process, for example, when you have a blocking library call or CPU intensive task.

wiremine 7y ago

Just want to say thank you to the Django core team. I was privileged to use Django full time for over 6 years. Both the project and the community are a joy. There's a few times I had to dive into Django's internals, and every time it was logical and straight forward to understand and modify. Thanks guys!

airstrike 7y ago

The community built around Django, coupled with the sheer amount of documentation and the stability of their APIs has done enough to forever ruin my expectations for any other coding project out there. Ostensibly everything else feels disorganized, poorly documented or rushly released. But the most commendable value of the DSF – which I believe is a model to be replicated – is how effectively it has managed to both attract new developers and also mentor them so well that they have gone on to become core developers who maintain the framework later on. I'm still subscribed to the django-developers mailing list, even if I have had no free time to contribute with more than a couple patches over many years, simply because I enjoy watching how incredibly smart and efficient developers collaborate. Kudos to all of you guys, and congrats on achieving this big milestone!

madrox 7y ago
I echo this sentiment exactly. I don't feel like OSS projects achieve this kind of success by accident. I'd love to see some write-up of some kind on how they do it. They're definitely the gold standard in my mind.
- frankwiles 7y ago
  That’s a damn good idea!

deusofnull 7y ago

i <3 django. thank you to all the people involved!

Alir3z4 7y ago

Django and Python have made me a much better developer. Using it for more than 10 years professionally, full time with no stop. It helped me to pay my bills, bring food to the table and build my ideas and get one step closer to my dreams. Thank you Django and everyone around or behind or even remotely related to it.

andy1729 7y ago
Can you explain lil bit more like what exactly you do with django (freelancing or job??) and your journey with django. I am sensing their is a great story which can inspire newbies like me! Thank you very much sir!
- Alir3z4 7y ago
  Prior to working with Django, I worked with ASP.NET/C# and later PHP with various web framework CakePHP, Zend, Symfony) and then worked Ruby on Rails for 2 years until I found Django (version 1.0.x I think) and from there in less than week, without even having a knowledge in Python, just by following tutorials and reading the source code I become productive and made production-ready apps. Almost all of my work with Django is through full time and once a while contracting gig here and there (very short ones.) Not just my day time job, I also implement all my side/personal/commercial projects with Django (gonevis.com is my latest project). Django never gave me a tough time to understand anything, for almost everything there's a well-maintained library and properly tested with enough community around it, it's ORM is crazy easy to work with, optimize, tweak and change when needed. Every new release is easy to upgrade, you may have some problem with third party libraries to catch up if something is not compatible with them but in my experience with medium-size code base (~300k LOC) the whole upgrade would take 2-3 hours and that was just changing code and running the tests, all in all, in 2-3 week most of the other third parties would have upgraded as well. It's mature framework, doesn't get crazy with new shiny things (NoSQL, MongoDB, WebSocket, etc), it might be late to some technology but it's because it stays in the corner until all these shiny things have worked out their problem and issues, then those things will become part of the official code base, otherwise if you're in hurry, there's always another third party that bring those for you, either NoSQL, WebSocket, push notifications, OTP, etc. For me, the most fascinating thing in Django is the ORM, Authentication, Admin, Views and template engine and the crazy support of PostgreSQL (almost the same applies to other database backends in Django), sessions, caching and every other single thing in that carefully have been layered upon each other and work in harmony. The database migration (formerly South migration then part of official) is something you can't easily find in other frameworks (Rails migration is great as well). Now, DRF is something else, since starting to work with DRF, I've become an API monster :D The framework that lets you implement your idea without getting in your way and allows you to bring ideas from zero to production very quickly, is the one I look forward to using (I use Java Spring once a while as well). For me, Django and it's surrounding makes sense, it's logical and it's not magical, maybe it's just me and it may not be something tasteful for others, if you find a framework you're comfortable with it, then use it, if you get into Django, I hope you feel the same.

andrewstuart 7y ago

I learned, in this order: bottle flask falcon django Django is incredible. It was useful to start with the minimalist frameworks to understand the true value of Django when I got there. Django is amongst my software top 10. Maybe top 5.

oehtXRwMkIs 7y ago
Curious as to what your top 10 is.

bobjordan 7y ago

Is substituting the Django ORM for SQLAlchemy doable in V3?

rptr_87 7y ago

I have been learning flask for development. Can someone describe advantages of using Django over flask?

ablekh 7y ago
Please check out the following posts: 1) https://devel.tech/features/django-vs-flask; https://devel.tech/features/django-vs-flask; 2) https://www.airpair.com/python/posts/django-flask-pyramid https://www.airpair.com/python/posts/django-flask-pyramid. Hope this helps.

aitchnyu 7y ago

OT, but what do you use to validate JSON posts without DRF? We mangled django form fields to validate a list of ids, but can I use forms to validate nested JSON? I now use Marshmallow for that use case but would love to stick with Django forms.

Hsoon 7y ago

Https://www.google.com