3 ms·

I've been looking at Python tasks queues recently. Does anyone have experience on how Celery and rq stack up? Rq is a lot smaller, more than 10x by line count.

by geertj 12y ago

I've been looking at Python tasks queues recently. Does anyone have experience on how Celery and rq stack up? Rq is a lot smaller, more than 10x by line count. So if it works just as well, I'd go with the simpler implementation.

xenator 12y ago
I used both, ended with Rq. Freedom if choice can be good, but when you able to make decision. Variety of backends, storages force you to understand how each component really work and when you dig into details you find that they all not equivalent. But you just need something f--kng working and you don't want to pay another guy to maintain zoo of different products. That is why I decided to use Rq, it is better to know limitations of something simple then know possibilities but not able to make choice.
- geertj 12y ago
  That's very helpful, thanks!
  - asksol 12y ago
    There are many differences, but most notably rq spawns one process per task. Line count is a stupid metric, e.g recently our line count doubled because of our new coding style, also the majority of the source code is tests.
SEJeff 12y ago
rq is like a luger pistol, light, simple, gets the job done. celery is like a .50 caliber machine gun, industrial strength, lots of options, used for a variety of completely different use cases. For simple stuff, use rq, but celery + rabbitmq work better if you have dozens and dozens plus worker nodes (ie: different servers), whereas with rq, you use redis, which could potentially be a SPOF, even with redis sentinel.
deleted 12y ago
[deleted]