3 ms·
Looks interesting. But shouldn't a library like celery work outside the context of a web framework? I don't see a reason to call this a distributed task queue '
by anuraggoel 17y ago
Looks interesting. But shouldn't a library like celery work outside the context of a web framework? I don't see a reason to call this a distributed task queue 'for Django' specifically, except for the dependencies on Django's ORM and settings definitions. Swapping out Django's ORM with SQLAlchemy (or DB-API) would make this project much more useful.
See pp (http://www.parallelpython.com/ http://www.parallelpython.com/) for something similar, without the django dependency. More parallel processing goodies at http://wiki.python.org/moin/ParallelProcessing http://wiki.python.org/moin/ParallelProcessing.
- gaborcselle 17y agoThe way I understand this, Celery is that binding glue between the Django a message queue (RabbitMQ), and cronjobs. I.e. you could swap out Django for CGI, and the ORM with SQLAlchemy, but then you might as well start from scratch.
- anuraggoel 17y agoI might not want to swap out Django with CGI, but I might want to use RabbitMQ-based distributed tasks in Pylons. Seems like a waste to write all that code from scratch. For Django users (and I am one), Celery is clearly useful. But a distributed task processing library, even if it is a thin wrapper over RabbitMQ, should not depend on a web-framework.
- asksol 17y agoYes. You're right, it could certainly be useful. But for now, it just hasn't been the focus to separate it from Django, yet. Tight deadlines and all that, you know. But it is definitely something I want to do, and you're more than welcome to help.
- ubernostrum 17y agoIt's actually not terribly hard. See this article for some pointers on how easy it is to work with AMQP from Python: http://playgroundblues.com/posts/2009/may/20/working-django-and-rabbitmq/ http://playgroundblues.com/posts/2009/may/20/working-django-... Meanwhile, I think there's nothing wrong with people distributing tools which integrate queuing solutions into specific libraries/frameworks; such things can often be quite useful and end up offering more natural interfaces for the task at hand.
- asksol 17y agoIt's not about communicating with AMQP. It uses carrot+pyamqplib for that.
- ubernostrum 17y agoI was simply replying to the seeming assertion that it's wrong to develop a queueing solution which integrates with Django. Writing queueing solutions in Python is easy, and integrating with popular tools should be fine.
- webology 17y agoCarrot (by the same author) might be more what you want - http://github.com/ask/carrot/ http://github.com/ask/carrot/ It started off as being Django specific then was later re-factored to not be.