6 ms·

What is Celery? Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports schedul

by jws 15y ago

What is Celery? Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

StavrosK 15y ago
Ask described it to me as "a library for discovering bugs in the multiprocessing library".
- asksol 15y ago
  Yes, but note that multiprocessing rocks. It's just that celery uses it so extensively :) (and now also eventlet/gevent if that suits better)
  - jnoller 15y ago
    And someone here has commit privs to fix those bugs too :) (ask does)
  - StavrosK 15y ago
    I agree with you on both counts. I've used multiprocessing to great effect, and, although it was fantastic for what it did, it was very buggy... Nowadays, I'd just use Celery instead!
reinhardt 15y ago
Does Celery support task dependencies? Would be great to specify a DAG of tasks and have them run in parallel. Basically something like Oozie but not restricted to Hadoop.
- dangoldin 15y ago
  If I understand your question correctly, you should be able to do this using callbacks.