5 ms·

Even more: why WSGI ? We need a django of the async world. We need something in Python to compete with nodejs, or even meteor, with async/await as primary tool

Even more: why WSGI ? We need a django of the async world. We need something in Python to compete with nodejs, or even meteor, with async/await as primary tool. And maybe a aWSGI support. But WSGI ? We have everything we need. They are proven tools. They work. Have great ecosystems and documentations. They do the job perfectly for non real time oriented web site. We don't need another one.

mephitix 9y ago
Seen Sanic[1] yet? I've become a fan... works well with asyncpg[2] too. [1]: https://github.com/channelcat/sanic https://github.com/channelcat/sanic [2]: https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg
- sametmax 9y ago
  It's very limited. We are far, far away from what django or meteor can provide. The API, while using async, is kinda old school (no plugins, namespaces, conf hooks...) and low level for some stuff (where is my task queue ?), and yet too high level for others (you give up control of the even loop policy completely). It's a nice attempt, but it's barely a draft IMO.
  - eric_bullington 9y ago
    When did you last use sanic? It's far from limited, and advancing quickly. However, it's a microframework, so it's true that you'll not get a full Django-like experience (and Meteor isn't even in the same class of web framework as Django or microframeworks like Sanic, so there's nothing to compare there either). But with respect to microframeworks, Sanic is getting pretty close to feature parity with Flask, only asynchronously. I'm not sure what kind of plugin architecture you're looking for, but you can do middleware with Sanic. Also custom protocols. Also decorators (e.g., for auth). And for namespacing, there are blueprints. I'm not sure what you mean by "conf hooks". It does help with Sanic if you're already familiar with Flask. Sanic is very close to an async version of Flask. For a task queue, you can run async tasks with Sanic alone using `app.add_task(some_async_function)`. It's also simple to use apscheduler if you want a more full-feature async scheduler. Example: from apscheduler.schedulers.asyncio import AsyncIOScheduler from sanic import Sanic app = Sanic() async def tick(): print('Work!') @app.listener('before_server_start') async def initialize_scheduler(app, loop): scheduler = AsyncIOScheduler({'event_loop': loop}) scheduler.add_job(tick, 'interval', seconds=1) scheduler.start() if __name__ == "__main__": app.run(host="0.0.0.0", port=8000, debug=True) Finally, unless things have changed, you should be able to set event loop policy with `asyncio.set_event_loop_policy`, have you tried it?
jkarneges 9y ago
> We need a django of the async world I would love to see this, BUT I don't think it's needed to compete with Node.js. Async is mostly interesting for networking, and you can get that now with Django Channels (https://channels.readthedocs.io/ https://channels.readthedocs.io/) or Pushpin (http://pushpin.org http://pushpin.org (disclosure: I'm the author)). Yeah they rely on separate processes which is a bit of a dodge, but IMHO the separation is good architecture.
- sametmax 9y ago
  I tried the django channels, and the way it works is a pita. Currently to setup a Django website you usually need a wsgi server, and a font-end server like nginx, then maybe a task queue and now your django channel server. None of them being well integrated. It's crazy. You should have one process that pops and manage the other ones, and allow them to integrate and communicate transparently, with not even a need for nginx/apache for small to medium websites. You should not have to setup a task queue manually, a thread/process pool and the like. You should have a way to communicate real time between the client code, the tasks in your queues, endpoints in your servers and all your workers. You should have synchronization primitives, a common configuration system (with live settings that propagates) and a clean way to build a hierarchy of components. The closest I get is crossbar.io serving django + celery and some custom code to hook them using the WAMP protocol. It's very, very nice, but far away from the integration we should have for those things in 2017. I have been starting to work on something (https://github.com/Tygs https://github.com/Tygs) but since it's a VERY hard problem, I struggle find continuity by working on it only here and there. This is something you need a few months full time to really dig deep. Especially since with async, the HTTP part is actually the easiest part. The hard part is to make it easily usable, configurable and debugable. Maybe there is a kickstarter to build for this.
  - amirouche 9y ago
    what about aiohttp? Doesn't it fit to the job of flask/django. A complete stack is one cookiecutter away.
    - 9y ago
mcless 9y ago
Aren't Flask, Django and Bottle also built on top of WSGI?
- aidos 9y ago
  I think that's what your parent is saying. There are good solutions built on WSGI, and WSGI itself is restrictive. Not to take anything away from this project.
  - mcless 9y ago
    This project too is a solution built on WSGI like Bottle and others. I agree plain WSGI is too restrictive and tedious to work with. So I don't understand how the parent comment's concerns about WSGI applies to this project but not to other WSGI-based frameworks.
    - detaro 9y ago
      There are already good solutions for WSGI. There are less for non-WSGI async options. The parents point is that if you now create something new, yet another WSGI solution doesn't add much to the general ecosystem, where a new async thing very well might.
      - mcless 9y ago
        That makes sense. Thanks for explaining.
  - sametmax 9y ago
    Exactly. If you build an async project, but do the same things than WSGI frameworks we gain nothing. We already have perfectly capable and productive frameworks. We use them everyday, we make money with it. They work. Async allows to maintain low latency permanent connections between each clients, being it a web page, a task queue, a server worker, etc. We finally can have it all connected. We can broadcast setting changes, propagate cache invalidation, push action notifications, update task completions to subscribers, allow everybody to react to stuff on the file system or the db instead of polling, all in soft real time. The potential is amazing. We all can do HTTP req / resp. The question now is: how good the tooling around this cycle is.
Spiritus 9y ago
What about apistar[1]? It recently added support for async/await and asyncio[2]. [1] https://github.com/encode/apistar https://github.com/encode/apistar [2] https://github.com/encode/apistar#asyncio https://github.com/encode/apistar#asyncio
- sametmax 9y ago
  Again it's very old school. You have async, you can do something incredible. You can setup real time RPC, propagate pub/sub accross all client and server nodes, provide a transparent tasks queue, enable live settings, etc. What you are doing here is just "what sync frameworks do, but async". There is no use for that. I can already do it with WSGI frameworks. It works well. It's robust, proven and productive. If you don't use async to provide novel features that only async enables, what's the point ?
giancarlostoro 9y ago
I just use CherryPy. It can run under WSGI, be a host for WSGI applications (and is usually used to host Flask applications) and runs it's own web server as well. CherryPy is pretty decent and performant, both Hulu and Netflix use it internally or have in the past. I do know one of the main devs is trying to make it support more of the asyncio stuff, though that might be a while. To answer your question as to "why WSGI?" the point is that if you write code that targets WSGI, no matter what you can put your Python code behind any WSGI compliant server. You wont be screwed because you picked a dying framework.
- sametmax 9y ago
  Yes but we have plenty of solutions for those. It's 2017, we got WSGI right. It's a done deal. Like trying to create another CGI framework.
  - giancarlostoro 9y ago
    I'm for WSGI, just stating that there are frameworks that fully support it properly as both hosts and end points for it.

mephitix 9y ago

Seen Sanic[1] yet? I've become a fan... works well with asyncpg[2] too. [1]: https://github.com/channelcat/sanic https://github.com/channelcat/sanic [2]: https://github.com/MagicStack/asyncpg https://github.com/MagicStack/asyncpg

sametmax 9y ago
It's very limited. We are far, far away from what django or meteor can provide. The API, while using async, is kinda old school (no plugins, namespaces, conf hooks...) and low level for some stuff (where is my task queue ?), and yet too high level for others (you give up control of the even loop policy completely). It's a nice attempt, but it's barely a draft IMO.
- eric_bullington 9y ago
  When did you last use sanic? It's far from limited, and advancing quickly. However, it's a microframework, so it's true that you'll not get a full Django-like experience (and Meteor isn't even in the same class of web framework as Django or microframeworks like Sanic, so there's nothing to compare there either). But with respect to microframeworks, Sanic is getting pretty close to feature parity with Flask, only asynchronously. I'm not sure what kind of plugin architecture you're looking for, but you can do middleware with Sanic. Also custom protocols. Also decorators (e.g., for auth). And for namespacing, there are blueprints. I'm not sure what you mean by "conf hooks". It does help with Sanic if you're already familiar with Flask. Sanic is very close to an async version of Flask. For a task queue, you can run async tasks with Sanic alone using `app.add_task(some_async_function)`. It's also simple to use apscheduler if you want a more full-feature async scheduler. Example: from apscheduler.schedulers.asyncio import AsyncIOScheduler from sanic import Sanic app = Sanic() async def tick(): print('Work!') @app.listener('before_server_start') async def initialize_scheduler(app, loop): scheduler = AsyncIOScheduler({'event_loop': loop}) scheduler.add_job(tick, 'interval', seconds=1) scheduler.start() if __name__ == "__main__": app.run(host="0.0.0.0", port=8000, debug=True) Finally, unless things have changed, you should be able to set event loop policy with `asyncio.set_event_loop_policy`, have you tried it?

jkarneges 9y ago

> We need a django of the async world I would love to see this, BUT I don't think it's needed to compete with Node.js. Async is mostly interesting for networking, and you can get that now with Django Channels (https://channels.readthedocs.io/ https://channels.readthedocs.io/) or Pushpin (http://pushpin.org http://pushpin.org (disclosure: I'm the author)). Yeah they rely on separate processes which is a bit of a dodge, but IMHO the separation is good architecture.

sametmax 9y ago
I tried the django channels, and the way it works is a pita. Currently to setup a Django website you usually need a wsgi server, and a font-end server like nginx, then maybe a task queue and now your django channel server. None of them being well integrated. It's crazy. You should have one process that pops and manage the other ones, and allow them to integrate and communicate transparently, with not even a need for nginx/apache for small to medium websites. You should not have to setup a task queue manually, a thread/process pool and the like. You should have a way to communicate real time between the client code, the tasks in your queues, endpoints in your servers and all your workers. You should have synchronization primitives, a common configuration system (with live settings that propagates) and a clean way to build a hierarchy of components. The closest I get is crossbar.io serving django + celery and some custom code to hook them using the WAMP protocol. It's very, very nice, but far away from the integration we should have for those things in 2017. I have been starting to work on something (https://github.com/Tygs https://github.com/Tygs) but since it's a VERY hard problem, I struggle find continuity by working on it only here and there. This is something you need a few months full time to really dig deep. Especially since with async, the HTTP part is actually the easiest part. The hard part is to make it easily usable, configurable and debugable. Maybe there is a kickstarter to build for this.
- amirouche 9y ago
  what about aiohttp? Doesn't it fit to the job of flask/django. A complete stack is one cookiecutter away.
  - 9y ago

mcless 9y ago

Aren't Flask, Django and Bottle also built on top of WSGI?

aidos 9y ago
I think that's what your parent is saying. There are good solutions built on WSGI, and WSGI itself is restrictive. Not to take anything away from this project.
- mcless 9y ago
  This project too is a solution built on WSGI like Bottle and others. I agree plain WSGI is too restrictive and tedious to work with. So I don't understand how the parent comment's concerns about WSGI applies to this project but not to other WSGI-based frameworks.
  - detaro 9y ago
    There are already good solutions for WSGI. There are less for non-WSGI async options. The parents point is that if you now create something new, yet another WSGI solution doesn't add much to the general ecosystem, where a new async thing very well might.
    - mcless 9y ago
      That makes sense. Thanks for explaining.
- sametmax 9y ago
  Exactly. If you build an async project, but do the same things than WSGI frameworks we gain nothing. We already have perfectly capable and productive frameworks. We use them everyday, we make money with it. They work. Async allows to maintain low latency permanent connections between each clients, being it a web page, a task queue, a server worker, etc. We finally can have it all connected. We can broadcast setting changes, propagate cache invalidation, push action notifications, update task completions to subscribers, allow everybody to react to stuff on the file system or the db instead of polling, all in soft real time. The potential is amazing. We all can do HTTP req / resp. The question now is: how good the tooling around this cycle is.

Spiritus 9y ago

What about apistar[1]? It recently added support for async/await and asyncio[2]. [1] https://github.com/encode/apistar https://github.com/encode/apistar [2] https://github.com/encode/apistar#asyncio https://github.com/encode/apistar#asyncio

sametmax 9y ago
Again it's very old school. You have async, you can do something incredible. You can setup real time RPC, propagate pub/sub accross all client and server nodes, provide a transparent tasks queue, enable live settings, etc. What you are doing here is just "what sync frameworks do, but async". There is no use for that. I can already do it with WSGI frameworks. It works well. It's robust, proven and productive. If you don't use async to provide novel features that only async enables, what's the point ?

giancarlostoro 9y ago

I just use CherryPy. It can run under WSGI, be a host for WSGI applications (and is usually used to host Flask applications) and runs it's own web server as well. CherryPy is pretty decent and performant, both Hulu and Netflix use it internally or have in the past. I do know one of the main devs is trying to make it support more of the asyncio stuff, though that might be a while. To answer your question as to "why WSGI?" the point is that if you write code that targets WSGI, no matter what you can put your Python code behind any WSGI compliant server. You wont be screwed because you picked a dying framework.

sametmax 9y ago
Yes but we have plenty of solutions for those. It's 2017, we got WSGI right. It's a done deal. Like trying to create another CGI framework.
- giancarlostoro 9y ago
  I'm for WSGI, just stating that there are frameworks that fully support it properly as both hosts and end points for it.