8 ms·

What's up Python? Django get background tasks, a new REPL, bye bye gunicorn

109 points by kevsamuel 2y ago

gonzo41 2y ago
Background tasks! I am effervescently happy to read this news.
geewee 2y ago
Django getting background tasks built-in is going to be really nice. It sucked having to manage celery or similar alongside it.
- nerdbaggy 2y ago
  You should check out Huey next time. It is a lifesaver for us. Allows us to do the tasks via SQLite db
  - geewee 2y ago
    Yeah that looks really promising. Haven't done Django as my daily driver for a few years now, and I don't recall stumbling upon it back then!
  - graemep 2y ago
    I use Huey too. Very simple and easy to manage. From a quick look the proposed tasks system seems similar to it.
daft_pink 2y ago
My real pain point as a user with Django is deployment. Python is so much easier to write, but javascript is so much easier to deploy. I wish they could make it easy to deploy django or other python framework in a serverless function with decent security baked in.
- 7bit 2y ago
  Where do you feel Django is harder to deploy? I have a different experience and think both are equally deployable.
ayhanfuat 2y ago
The new REPL looks great. Indeed qtconsole on steroid. Edit: Ooops. IPython magics don't seem to work. I hope they add it.
mg 2y ago
What I love about Django is that you can create a Django project with just one file. You can turn a fresh Debian machine into a running Django web app by just doing: apt install -y python3-django apache2 libapache2-mod-wsgi-py3 And then creating the one file Django needs: /var/www/mysite/mysite/wsgi.py: import os import django from django.core.wsgi import get_wsgi_application os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi') application = get_wsgi_application() ROOT_URLCONF = 'mysite.wsgi' SECRET_KEY = 'hello' def index(request): return django.http.HttpResponse('This is the homepage') def cats(request): return django.http.HttpResponse('This is the cats page') urlpatterns = [ django.urls.path('', index), django.urls.path('cats', cats), ] And then telling Apache where to look for it: /etc/apache2/sites-enabled/000-default.conf: ServerName 127.0.0.1 WSGIPythonPath /var/www/mysite <VirtualHost *:80> WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py <Directory /var/www/mysite/mysite> <Files wsgi.py> Require all granted </Files> </Directory> </VirtualHost> And voilá, you have a running Django application, which you can expand upon to any size and complexity. If you want to try it in a docker container, you can run docker run -it --rm -p80:80 debian:12 perform the steps above and then access the Django application at 127.0.0.1
- devwastaken 2y ago
  This is one of the core criticisms of Django and python web backends. WSGI is old tech that in your example requires extra packages and code in project to work. By now Python should stand on its own and handle this independently, being proxied to by better http servers like nginx or caddy.
  - ranger_danger 2y ago
    > This is one of the core criticisms of Django and python web backends. No it's not. Please cite reputable sources.
    - devwastaken 2y ago
      One does not need to cite a source of criticism, it is in itself the criticism.
wormlord 2y ago
Oh man, I wish this was a bit more mature. I am writing a distributed image processing app right now with Celery, Flask, and SQLAlchemy, but I am finding it difficult to KISS. I was tempted to use Django but couldn't justify the overhead. If they get this cleaned up in the next year that would be awesome!
- dacryn 2y ago
  justify the overhead? But you are using flask and sqlalchemy? You made the wrong choice I would guess. Django does not have 'overhead' to justify in that scenario.
  - Paul-Craft 2y ago
    Here's how much is installed when you do `pip install django` currently: 122370 django python=103934,javascript=18407,xml=29 2734 sqlparse python=2734 1110 asgiref python=1110 This will probably change when `django-tasks` gets merged in. Here's what installing `flask` + `sqlalchemy` pulls in: 141314 sqlalchemy python=141314 11464 werkzeug python=11159,javascript=305 8934 greenlet cpp=4993,python=2729,ansic=1092,asm=120 8606 jinja2 python=8606 5842 click python=5842 3907 flask python=3907 2036 top_dir python=2036 644 itsdangerous python=644 487 markupsafe ansic=278,python=209 349 blinker python=349 Looks like the same order of magnitude to me, which is surprising. I had always thought of `django` as being heavy weight and opinionated, while `flask` + `sqlalchemy` was light and modular. All numbers generated using David A. Wheeler's 'SLOCCount'. :-)
    - evantbyrne 2y ago
      I suspect that people come to the conclusion that Django has performance issues by looking at synthetic benchmark rankings between frameworks. In that sense it may or may not be relatively slow, but it is probably fast-enough for almost all web services in-practice. And if you measure with all of the speediest HTTP interfaces and find out it isn't fast-enough, then frankly Python is probably not the right choice at all.
pgjones 2y ago
Quart, the ASGI version of Flask, also has background tasks, https://quart.palletsprojects.com/en/latest/how_to_guides/background_tasks.html https://quart.palletsprojects.com/en/latest/how_to_guides/ba..., and there is an extension Quart-Tasks, https://github.com/pgjones/quart-tasks https://github.com/pgjones/quart-tasks, to run them on a schedule. Via @tasks.cron("*/5 * * * *") # every 5 minutes async def infrequent_task(): ... In addition Hypercorn, https://github.com/pgjones/hypercorn https://github.com/pgjones/hypercorn, (a ASGI and WSGI) also does not use gunicorn, having migrated many years ago.
mstaoru 2y ago
Background tasks look well thought out, looking forward to try that. Despite being somewhat out of fashion, Python and Django continue to serve more down to earth software development well after so many years. I picked it up around 2007 (remember the "magic removal" branch), and it only got better ever since. Many call Python "toy language" or "data scientist language" but the truth is that 99.9% of modern web development is database-backed and, unless you have some strictly computational tasks behind your APIs, the fact that e.g. Rust is 100x "faster" than Python is mostly meaningless. Django just works. Of course, there can be different personal preferences, but I'm really amazed by the project that is closing in on 20 years that is still absolutely relevant and a joy to work with.
- boyka 2y ago
  I've been looking for know anything resembling django's ORM _including_ migrations in rust, but haven't found a good replacement.
deleted 2y ago
[deleted]
threecheese 2y ago
Love Django, but the issue we’ve had with tasks isn’t with performance, scalability, or devex - it’s with operations. Django-tasks afaict has the same issues as Celery/others in that there’s no Django Admin-like tool that reliably allows you to manage task execution, specifically failures. Without this, I can’t use async tasks for anything that’s important to the business or data consistency generally - which means I don’t use it at all, favoring “normal” queuing backends. Is Django-tasks the solution that would permit this to be built?
- SCUSKU 2y ago
  Second this. I’m amazed by the lack of any Django admin interface to start a task or see how it failed, it seems like such an obvious use case. I’ve tried Celery Flower but I found it lacking. I’m surprised because it seems like such a common use case, and given the size of the Django community I would’ve expected something to exist…
mmarian 2y ago
Hah, I just started using Django and was in need of background tasks!
develatio 2y ago
I have been doing Django for most of the last 10 years and I love it. I love Python. I love how well made are most of its core parts. And I love how some 3rd party packages (DRF aka django-rest-framework, django-filter, django-storages) fill the "missing" gaps. That said, boy does Django must change it's current strategy... RoR and Laravel have been getting all sort of goodies baked in the framework itself such as tasks queue, nice admin features, integration with frontend technologies (scss, js, even jsx/tsx), even entire packages to manage payment systems, mailing and so on... Good quality features that are ready to use and are integrated inside the framework, which easy to use APIs and fantastic admin integration. Django devs, on the other hand, have been (imho) ignoring the users (devs) and have been denying any and all attempts at bringing something similar to Django. There are at least half a dozen packages for each "thing" that you want to do, and some are in a very good shape, but others are just hopelessly broken. What's even worse, because they're not part of Django, most of the time there are some edge cases that just can't be implemented without subclassing Django's core. One of the most recent examples being Django's support for async operations. Except that Django itself doesn't provide a nice way to create / expose an API, which is why people use DRF. But DRF doesn't support async, which is why you need to add "adrf" (https://github.com/em1208/adrf https://github.com/em1208/adrf) to DRF. But adrf doesn't support async generic viewsets, so you must add aiodrf (https://github.com/imgVOID/aiodrf https://github.com/imgVOID/aiodrf) to adrf. But aiodrf doesn't support (...) you get the point. You end up in a situation in which you're pilling packages on top of packages on top of packages just so you can have an asynchronous API. Madness. Supporting scss it another example of pilling packages on top of packages. It just never ends... I can't express the desire I have for Django to start integrating packages into its core and get on par with RoR and Laravel. /rant
- 7bit 2y ago
  DRF is cancer. Try doing anything simple with DRF and you got subclasses over subclasses with overrides and overrides. I hate DRF with all my heart. DRF works only, if you intend to exactly match your model. If deviate just a little bit, the complexity explodes in hour hand. The biggest gripe I have with Django is that coupling it with modern UIs like Svelte and company is massively complex. The community has failed to deliver a clear guidance on how to handle implementation of such frameworks, including Vite and build steps. Which is the primary thing that drives me away from Django. I'm looking towards Laravel next. Very often, when researching how to implement JS Frameworks into Django I seem to find guidance on how to do this with Laravel and I want to know if it's really that comfy as it sounds.

109 points by kevsamuel 2y ago

Background tasks! I am effervescently happy to read this news.

Django getting background tasks built-in is going to be really nice. It sucked having to manage celery or similar alongside it.

nerdbaggy 2y ago
You should check out Huey next time. It is a lifesaver for us. Allows us to do the tasks via SQLite db
- geewee 2y ago
  Yeah that looks really promising. Haven't done Django as my daily driver for a few years now, and I don't recall stumbling upon it back then!
- graemep 2y ago
  I use Huey too. Very simple and easy to manage. From a quick look the proposed tasks system seems similar to it.

daft_pink 2y ago

My real pain point as a user with Django is deployment. Python is so much easier to write, but javascript is so much easier to deploy. I wish they could make it easy to deploy django or other python framework in a serverless function with decent security baked in.

7bit 2y ago
Where do you feel Django is harder to deploy? I have a different experience and think both are equally deployable.

ayhanfuat 2y ago

The new REPL looks great. Indeed qtconsole on steroid. Edit: Ooops. IPython magics don't seem to work. I hope they add it.

mg 2y ago

What I love about Django is that you can create a Django project with just one file. You can turn a fresh Debian machine into a running Django web app by just doing: apt install -y python3-django apache2 libapache2-mod-wsgi-py3 And then creating the one file Django needs: /var/www/mysite/mysite/wsgi.py: import os import django from django.core.wsgi import get_wsgi_application os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi') application = get_wsgi_application() ROOT_URLCONF = 'mysite.wsgi' SECRET_KEY = 'hello' def index(request): return django.http.HttpResponse('This is the homepage') def cats(request): return django.http.HttpResponse('This is the cats page') urlpatterns = [ django.urls.path('', index), django.urls.path('cats', cats), ] And then telling Apache where to look for it: /etc/apache2/sites-enabled/000-default.conf: ServerName 127.0.0.1 WSGIPythonPath /var/www/mysite <VirtualHost *:80> WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py <Directory /var/www/mysite/mysite> <Files wsgi.py> Require all granted </Files> </Directory> </VirtualHost> And voilá, you have a running Django application, which you can expand upon to any size and complexity. If you want to try it in a docker container, you can run docker run -it --rm -p80:80 debian:12 perform the steps above and then access the Django application at 127.0.0.1

devwastaken 2y ago
This is one of the core criticisms of Django and python web backends. WSGI is old tech that in your example requires extra packages and code in project to work. By now Python should stand on its own and handle this independently, being proxied to by better http servers like nginx or caddy.
- ranger_danger 2y ago
  > This is one of the core criticisms of Django and python web backends. No it's not. Please cite reputable sources.
  - devwastaken 2y ago
    One does not need to cite a source of criticism, it is in itself the criticism.

wormlord 2y ago

Oh man, I wish this was a bit more mature. I am writing a distributed image processing app right now with Celery, Flask, and SQLAlchemy, but I am finding it difficult to KISS. I was tempted to use Django but couldn't justify the overhead. If they get this cleaned up in the next year that would be awesome!

dacryn 2y ago
justify the overhead? But you are using flask and sqlalchemy? You made the wrong choice I would guess. Django does not have 'overhead' to justify in that scenario.
- Paul-Craft 2y ago
  Here's how much is installed when you do `pip install django` currently: 122370 django python=103934,javascript=18407,xml=29 2734 sqlparse python=2734 1110 asgiref python=1110 This will probably change when `django-tasks` gets merged in. Here's what installing `flask` + `sqlalchemy` pulls in: 141314 sqlalchemy python=141314 11464 werkzeug python=11159,javascript=305 8934 greenlet cpp=4993,python=2729,ansic=1092,asm=120 8606 jinja2 python=8606 5842 click python=5842 3907 flask python=3907 2036 top_dir python=2036 644 itsdangerous python=644 487 markupsafe ansic=278,python=209 349 blinker python=349 Looks like the same order of magnitude to me, which is surprising. I had always thought of `django` as being heavy weight and opinionated, while `flask` + `sqlalchemy` was light and modular. All numbers generated using David A. Wheeler's 'SLOCCount'. :-)
  - evantbyrne 2y ago
    I suspect that people come to the conclusion that Django has performance issues by looking at synthetic benchmark rankings between frameworks. In that sense it may or may not be relatively slow, but it is probably fast-enough for almost all web services in-practice. And if you measure with all of the speediest HTTP interfaces and find out it isn't fast-enough, then frankly Python is probably not the right choice at all.

pgjones 2y ago

Quart, the ASGI version of Flask, also has background tasks, https://quart.palletsprojects.com/en/latest/how_to_guides/background_tasks.html https://quart.palletsprojects.com/en/latest/how_to_guides/ba..., and there is an extension Quart-Tasks, https://github.com/pgjones/quart-tasks https://github.com/pgjones/quart-tasks, to run them on a schedule. Via @tasks.cron("*/5 * * * *") # every 5 minutes async def infrequent_task(): ... In addition Hypercorn, https://github.com/pgjones/hypercorn https://github.com/pgjones/hypercorn, (a ASGI and WSGI) also does not use gunicorn, having migrated many years ago.

mstaoru 2y ago

Background tasks look well thought out, looking forward to try that. Despite being somewhat out of fashion, Python and Django continue to serve more down to earth software development well after so many years. I picked it up around 2007 (remember the "magic removal" branch), and it only got better ever since. Many call Python "toy language" or "data scientist language" but the truth is that 99.9% of modern web development is database-backed and, unless you have some strictly computational tasks behind your APIs, the fact that e.g. Rust is 100x "faster" than Python is mostly meaningless. Django just works. Of course, there can be different personal preferences, but I'm really amazed by the project that is closing in on 20 years that is still absolutely relevant and a joy to work with.

boyka 2y ago
I've been looking for know anything resembling django's ORM _including_ migrations in rust, but haven't found a good replacement.

deleted 2y ago

[deleted]

threecheese 2y ago

Love Django, but the issue we’ve had with tasks isn’t with performance, scalability, or devex - it’s with operations. Django-tasks afaict has the same issues as Celery/others in that there’s no Django Admin-like tool that reliably allows you to manage task execution, specifically failures. Without this, I can’t use async tasks for anything that’s important to the business or data consistency generally - which means I don’t use it at all, favoring “normal” queuing backends. Is Django-tasks the solution that would permit this to be built?

SCUSKU 2y ago
Second this. I’m amazed by the lack of any Django admin interface to start a task or see how it failed, it seems like such an obvious use case. I’ve tried Celery Flower but I found it lacking. I’m surprised because it seems like such a common use case, and given the size of the Django community I would’ve expected something to exist…

mmarian 2y ago

Hah, I just started using Django and was in need of background tasks!

develatio 2y ago

I have been doing Django for most of the last 10 years and I love it. I love Python. I love how well made are most of its core parts. And I love how some 3rd party packages (DRF aka django-rest-framework, django-filter, django-storages) fill the "missing" gaps. That said, boy does Django must change it's current strategy... RoR and Laravel have been getting all sort of goodies baked in the framework itself such as tasks queue, nice admin features, integration with frontend technologies (scss, js, even jsx/tsx), even entire packages to manage payment systems, mailing and so on... Good quality features that are ready to use and are integrated inside the framework, which easy to use APIs and fantastic admin integration. Django devs, on the other hand, have been (imho) ignoring the users (devs) and have been denying any and all attempts at bringing something similar to Django. There are at least half a dozen packages for each "thing" that you want to do, and some are in a very good shape, but others are just hopelessly broken. What's even worse, because they're not part of Django, most of the time there are some edge cases that just can't be implemented without subclassing Django's core. One of the most recent examples being Django's support for async operations. Except that Django itself doesn't provide a nice way to create / expose an API, which is why people use DRF. But DRF doesn't support async, which is why you need to add "adrf" (https://github.com/em1208/adrf https://github.com/em1208/adrf) to DRF. But adrf doesn't support async generic viewsets, so you must add aiodrf (https://github.com/imgVOID/aiodrf https://github.com/imgVOID/aiodrf) to adrf. But aiodrf doesn't support (...) you get the point. You end up in a situation in which you're pilling packages on top of packages on top of packages just so you can have an asynchronous API. Madness. Supporting scss it another example of pilling packages on top of packages. It just never ends... I can't express the desire I have for Django to start integrating packages into its core and get on par with RoR and Laravel. /rant

7bit 2y ago
DRF is cancer. Try doing anything simple with DRF and you got subclasses over subclasses with overrides and overrides. I hate DRF with all my heart. DRF works only, if you intend to exactly match your model. If deviate just a little bit, the complexity explodes in hour hand. The biggest gripe I have with Django is that coupling it with modern UIs like Svelte and company is massively complex. The community has failed to deliver a clear guidance on how to handle implementation of such frameworks, including Vite and build steps. Which is the primary thing that drives me away from Django. I'm looking towards Laravel next. Very often, when researching how to implement JS Frameworks into Django I seem to find guidance on how to do this with Laravel and I want to know if it's really that comfy as it sounds.