4 ms·
Why should I use this over the simplier RQ?
by reubano 10y ago
Why should I use this over the simplier RQ?
- joowani 10y agoRQ is certainly useful (with finer control over messages), but we've been having a lot of problems with it lately in production due to it being memory bound (Redis). Improper code deploys or insufficient/stuck workers would quickly explode the queues and make the broker go oom in matter of hours. Memory is also very expensive. With KQ/Kafka I was hoping it would provide us with a lot more to buffer for human errors and scale better.
- reubano 10y agoOk, that makes sense. I've never dealt with the scale you are talking about so have yet to run into these issues with redis. It may be best to put this info in the readme so people can decide for themselves when to use RQ or KQ.
- brianwawok 10y agoOr the proven Celery
- dozzie 10y agoThis Celery? https://news.ycombinator.com/item?id=12844206 https://news.ycombinator.com/item?id=12844206
- brianwawok 10y agoI mean, Celery is a pretty amazing library. It has flaws, but I doubt this new project has no flaws. They are just unknown still. For that specific bug in question, does KQ have logic around DST switch? I was not able to find any. So it looks like you would have the same bug in KQ, right?
- al2o3cr 10y agoDunno why people would blame the Celery devs for that bug - if you're setting the clock on production servers to a non-monotonic TZ you are going to have a bad time.
- dozzie 10y agoSo you basically blame people for living in countries that use daylight saving time? O_O Also, I would understand if it were tasks between 2AM and 3AM that run twice in transition from summer time to winter time (there are two 2:30 that day, after all), but the bug affected tasks scheduled for the middle of the day.