6 ms·
Celery 3.0 has been released
- hghghg 14y agoi don´t even understand what this celery code is used for, and their web page can´t explain it either.
- level09 14y agoBrilliant, I have started to use celery a few months ago, now we a use it to back our video transcoding and automation platform, our search engine, and our mobile app backend. Excellent job celery team !
- bryanh 14y agoCelery is one of those rare libraries where improvements to itself are improvements to my sanity and well being. That said, I did run into an issue with periodic tasks (our bread and butter at Zapier). Detailed here: https://github.com/celery/celery/issues/844 https://github.com/celery/celery/issues/844 Thanks a bunch Ask.
- topbanana 14y agoIs it compatible with Hummus 2.0?
- creamyhorror 14y agoYou say that now, but tomorrow Hummus.py 0.5 will be announced.
- stevvooe 14y agoOnly peanut butter 1.0, and there may be legacy support for ranch3, but you'll have to check the docs.
- fatiherikli 14y agochain is amazing.
- rox 14y agoFinally I could use officially supported 'Canvas' to design workflows, before that I had to use the celery-tasktree package.
- srj55 14y agolooks like redis is getting improved support in this release. Anyone using redis instead of rabbitmq?
- ergo14 14y agoI do for Errormator task queue, since 2.5, no issues at all.
- misiti3780 14y agoim using redis instead of rabbitmq - have had zero problems so far. great work, this project is awesome
- conesus 14y agoAre you using redis with slaves? When I do that, celery throws errors when trying to add tasks to my redis queue.
- asksol 14y agowhat's the error and traceback? (paste at pastie.org or similar)
- conesus 14y agoPreface this by saying that I've been successfully and happily using celery since 0.8, which was over 2 years ago. Since 2.0, it's been rock solid. But then I switched from RabbitMQ to redis and when I added redis slaves, it broke. Here goes: http://pastie.org/4216467 http://pastie.org/4216467 It's probably due to redis' INFO command giving different values. Now would be a great time to get it sorted out, too.
- asksol 14y agoAbout your error, seems strange that a simple info command gives that error, maybe you're running an outdated version of redis-py/redis-server?
- conesus 14y agoLatest redis (2.4.15) and using redis-py 2.4.10. Just checked the CHANGELOG for redis-py, which is now at 2.4.13, and voila: * 2.4.11 * Made the INFO command more tolerant of Redis changes formatting. Fix for #217. Doh! Off by a single patch release. Thanks for the help. See, folks? Celery is amazing in multiple ways.
- misiti3780 14y ago
- hymloth 14y agoIt seems that the Eventlet pool does not work with the celery umbrella command. With celeryd it works ok. Is that true or do I have misconfigured something?
- asksol 14y agocould be a bug, could you please open an issue at http://github.com/celery/celery/issues http://github.com/celery/celery/issues?
- rwhitman 14y agoI like the Autechre album based version naming convention
- Herbert2 14y agoJust wanted to say thanks to Ask for Celery. Absolutely fantastic for distributed systems.
- rjurney 14y agoDoes Celery still create a new queue for every worker on rabbitmq? This bizarre behavior makes it hard to use Celery with amqp :(
- asksol 14y agoone queue for every worker instance, yes! this is for broadcasting remote control commands, but I don't see how this poses a problem? If you mean the one queue for every task behavior of the amqp result backend then that is also a yes. Usually with replies in amqp you create one queue for every client, but Celery is often used in a web context where the process that initiated the task may not be the same process that collects the reply, so this is why it uses one queue for every task (it's also documented). There's an experimental result backend for RPC-style replies, that uses transient messages and one queue per client: http://github.com/celery/celery/tree/kombuRPC http://github.com/celery/celery/tree/kombuRPC
- rjurney 14y agoThat is a problem for me, because I want all events of one type running through one AMQP queue to monitor throughput, etc.
- asksol 14y agoSeveral ways to accomplish that, but I would recommend using kombu in combination with celery to have the task manually send messages. You could set the exchange type of the results exchange to be topic too, that way you would both have result queues and you could additionally bind a queue to the results exchange to get a copy of all the messages sent there. But if you don't need to listen for individual results then I'd rather just send messages manually. You have both connection and producer pools in Celery, so it's rather convenient to combine kombu with celery.
- reinhardt 14y ago> Over 600 commits, 30k additions/36k deletions. The large number of deletions is a pretty good sign of the project's health. Congrats!
- JimmyL 14y agoCelery has it's quirks now and then (for reasons I can't nail down or reliably reproduce, enabling CELERYD_FORCE_EXECV breaks a lot of things), but I consider Ask and the rest of the #celery gang to be models for how to have a welcoming community around an open-source project. Looking forwards to experimenting with the non-multiprocessing worker - something in my current setup regularly leaks memory, and for the above-mentioned reason I can't automatically recycle worker processes to clean it up.
- simonpantzare 14y agoI find this[1] example in Getting Started/Next Steps to be backwards: # incomplete partial: add(?, 2) >>> s2 = add.s(2) # resolves the partial: add(8, 2) >>> res = s2.delay(8) >>> res.get() 10 Shouldn't it behave like functools.partial[2]? >>> import functools >>> abc = lambda a, b, c: (a, b, c) >>> bc = functools.partial(abc, "a") >>> bc("b", "c") ('a', 'b', 'c') 1. http://docs.celeryproject.org/en/latest/getting-started/next-steps.html#and-there-s-that-calling-api-again http://docs.celeryproject.org/en/latest/getting-started/next... 2. http://docs.python.org/library/functools.html#functools.partial http://docs.python.org/library/functools.html#functools.part...
- asksol 14y agoVery good question! The reason is that I found that it better matches what you use them for in Celery, since they are used e.g. to forward results from the previous tasks to a callback. Often you have a signatures like: def blur_image(image, amount=1), where the partial is used like render_image.s(file) | blur_image.s() functools.partial is usually used the other way; it's most common to create functools.partials where the first argument is satisfied.
- gonvaled 14y agoCan somebody compare celery to beanstalk?