4 ms·

Show HN: I'm trying to bring Async to Django ORM

6 points by privacyonsec 5y ago

uniqueuid 5y ago
Thanks, it's great to see Django adding slowly but surely in all sorts of places. By the way, have you looked at tortoise orm, which is a fully async python ORM with very Django-like syntax? [1] [1] https://github.com/tortoise/tortoise-orm https://github.com/tortoise/tortoise-orm
- allen-munsch 5y ago
  for an ongoing discussion see: https://forum.djangoproject.com/t/asynchronous-orm/5925/46 https://forum.djangoproject.com/t/asynchronous-orm/5925/46
- wraptile 5y ago
  Not OP but I've been experimenting with tortoise and it's pretty awesome! I love that it works with pydantic out of the box so it integrates with FastApi really nicely. I just started a small side project with fastapi + tortoise + postgresql + vuejs and so far it's a pretty fun and smooth experience!
- privacyonsec 5y ago
  Indeed, but the goal of the project is to support django ORM, if you already have a backend running under django and you wanna migrate to async, you will have to write a lot of wrappers. with this module it patches django (QuerySet) with async methods. You still have to update your code for example MyModel.objects.get(name="something") becomes MyModel.objects.async_get(name="something") so even though tortoise orm syntax's is similar to django, everything else can be different, migrations etc ...
samwillis 5y ago
Awesome! For those of us a little out of the loop, is this implementation being developed with guidance from the core Django devs with a view to adopting into core? (I tried to find any references in the Django forums but may have missed them) I believe they intend to have native support for async with the ORM, but it’s a monumental feature to add with lots of edge cases. I believe about 50% of Django is it’s ORM.
- allen-munsch 5y ago
  I'm not sure if its official. Would love some more guidance/clarity/docs/funding from the django foundation on what it looks like to migrate legacy code to the new ways. The rednaks repo works great for just giving the new async stuff a go. If everything else is also using async. I did some experimentation with this. And its a pain trying to migrate a production application that uses gevent and psycogreen2. The documentation on the code migration path is pretty sparse. The main hiccup that I ran into was psycogreen2 not being supported. https://stackoverflow.com/questions/67735453/django-async-orm-sync-to-async-executor-swap-out-concurrent-futures-threadpoolex https://stackoverflow.com/questions/67735453/django-async-or... https://github.com/rednaks/django-async-orm/discussions/9 https://github.com/rednaks/django-async-orm/discussions/9 https://github.com/rednaks/django-async-orm/discussions/6 https://github.com/rednaks/django-async-orm/discussions/6 Everything works great if everything is already using the regular threading and asyncio/async methods, or JUST gevent/psycogreen. The django documentation suggests mounting WSGI into ASGI. But, it breaks down immediately going between the two paradigms. I put a little experiment together a bit ago to see what would work in production, and my results. https://github.com/allen-munsch/benchmark-django-fastapi https://github.com/allen-munsch/benchmark-django-fastapi

6 points by privacyonsec 5y ago

Thanks, it's great to see Django adding slowly but surely in all sorts of places. By the way, have you looked at tortoise orm, which is a fully async python ORM with very Django-like syntax? [1] [1] https://github.com/tortoise/tortoise-orm https://github.com/tortoise/tortoise-orm

allen-munsch 5y ago
for an ongoing discussion see: https://forum.djangoproject.com/t/asynchronous-orm/5925/46 https://forum.djangoproject.com/t/asynchronous-orm/5925/46
wraptile 5y ago
Not OP but I've been experimenting with tortoise and it's pretty awesome! I love that it works with pydantic out of the box so it integrates with FastApi really nicely. I just started a small side project with fastapi + tortoise + postgresql + vuejs and so far it's a pretty fun and smooth experience!
privacyonsec 5y ago
Indeed, but the goal of the project is to support django ORM, if you already have a backend running under django and you wanna migrate to async, you will have to write a lot of wrappers. with this module it patches django (QuerySet) with async methods. You still have to update your code for example MyModel.objects.get(name="something") becomes MyModel.objects.async_get(name="something") so even though tortoise orm syntax's is similar to django, everything else can be different, migrations etc ...

samwillis 5y ago

Awesome! For those of us a little out of the loop, is this implementation being developed with guidance from the core Django devs with a view to adopting into core? (I tried to find any references in the Django forums but may have missed them) I believe they intend to have native support for async with the ORM, but it’s a monumental feature to add with lots of edge cases. I believe about 50% of Django is it’s ORM.

allen-munsch 5y ago
I'm not sure if its official. Would love some more guidance/clarity/docs/funding from the django foundation on what it looks like to migrate legacy code to the new ways. The rednaks repo works great for just giving the new async stuff a go. If everything else is also using async. I did some experimentation with this. And its a pain trying to migrate a production application that uses gevent and psycogreen2. The documentation on the code migration path is pretty sparse. The main hiccup that I ran into was psycogreen2 not being supported. https://stackoverflow.com/questions/67735453/django-async-orm-sync-to-async-executor-swap-out-concurrent-futures-threadpoolex https://stackoverflow.com/questions/67735453/django-async-or... https://github.com/rednaks/django-async-orm/discussions/9 https://github.com/rednaks/django-async-orm/discussions/9 https://github.com/rednaks/django-async-orm/discussions/6 https://github.com/rednaks/django-async-orm/discussions/6 Everything works great if everything is already using the regular threading and asyncio/async methods, or JUST gevent/psycogreen. The django documentation suggests mounting WSGI into ASGI. But, it breaks down immediately going between the two paradigms. I put a little experiment together a bit ago to see what would work in production, and my results. https://github.com/allen-munsch/benchmark-django-fastapi https://github.com/allen-munsch/benchmark-django-fastapi