4 ms·
Well there are a couple drawbacks. The author is describing celery as a solution to the GIL thread lock problem. However the time it takes to dispatch the job[1
by denom 10y ago
Well there are a couple drawbacks. The author is describing celery as a solution to the GIL thread lock problem. However the time it takes to dispatch the job[1] and fire up the worker is much greater than the time it takes to create a thread. So while celery is _a_ solution to the problem, it's not an equivalent solution. Just something to be aware of.
The type of computation you're modeling is limited by the async/dispatched nature of celery. Threads can communicate data structures to each other. With celery, that's awkward with the latency and async semantics.
[1] I've used rabbitmq to dispatch jobs over to celery in the past, here's a good overview of latency in that process. TLDR requests that hit the wire are on the order of milliseconds: https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/ https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory...
- kerkeslager 10y agoI guess I'm just not sure what people are expecting here. The point of Celery isn't to parallelize and synchronize, it's to fire off a task, probably on an entirely different machine, and forget about it. If you're trying to sync up afterward, you're going to run into problems because it's not designed for that.