3 ms·
Having just hacked together an ugly threaded task queue for scraping and multi-stage data processing in Django, this looks like a breath of fresh air. I need to
by pie 17y ago
Having just hacked together an ugly threaded task queue for scraping and multi-stage data processing in Django, this looks like a breath of fresh air. I need to work my way out of the self-inflicted mess I've created.
Does anyone have experience with this library or anything similar?
- conesus 17y agoYeah, I also have experience in hacking together a multi-threaded task queue with ugly results. Try getting messaging in multiple daemon threads to communicate with the web client (who is spawned off early in the process, only to be reunited later), and you'll see how much of a bear this is. It's not too hard to perform only one of the use cases that celery handles, but to get all of them? I'm installing celery this weekend and will see how it goes. Maybe if it goes well, I can give a before-and-after blog post and submit it to HN.