3 ms·

Actually with uWSGI, be in on Flask, Django or else: I don't need neither Celery nor Redis. uWSGI has a built-in celery-ish spooler, cron-ish task scheduler, me

Actually with uWSGI, be in on Flask, Django or else: I don't need neither Celery nor Redis. uWSGI has a built-in celery-ish spooler, cron-ish task scheduler, memcache-ish key-value store, along with plenty of other toys that I love from the deepest of my heart ... And I've been in this (great) situation for years, not planning to move out to more complicated stacks. I would highly recommend uWSGI over of Celery and Redis, which I used in the past, prior to doing it all in uWSGI, unless you have a really good reason which I'm eager to read about. And now that uWSGI supports a lot of languages, even if I have some PHP or whatever to deploy I'll go for uWSGI, one of the most beautiful piece of software I have the chance to use.

d33 7y ago
Could you elaborate, e.g. link to documentation, subprojects or example code? I'd love to get rid of Celery because of how difficult it is for me to tweak it for good performance.
- dzonga 7y ago
  you can also use dramatiq
- darkerside 7y ago
  Would love to see more detail as well because I'm a bit skeptical. From uWSGI documentation, it looks like to coordinate cron across anything more than a single server setup, you'd need to configure a Legion, which means you're then integrating uWSGI's orchestration framework with whatever you're already using (k8, ECS, etc). I like minimalism, but sometimes batteries are included for a reason.
  - GlennS 7y ago
    I suspect they are using a single server. I found uwsgi's mules and cache2 very useful in that situation. If someone finds that Redis and Celery are more complication than they need for a given task, then I think they're probably not using an orchestration framework.
- GlennS 7y ago
  This is what I have used in the past. I think they're very convenient if you are just running one instance of uwsgi and want to share some state quickly without going through a persistent database. If you're worrying about tweaking Celery for performance, then I suspect your uses may be a bit more complex than uwsgi's mules are designed for though. https://uwsgi-docs.readthedocs.io/en/latest/Caching.html https://uwsgi-docs.readthedocs.io/en/latest/Caching.html https://uwsgi-docs.readthedocs.io/en/latest/Mules.html https://uwsgi-docs.readthedocs.io/en/latest/Mules.html The cache is a simple key-value store. Works well. I later swapped it out for Redis, because I needed to shared my cache between multiple machines. Switching to Redis was a very quick and easy replacement, so you don't need to worry about lock in. I used mules in a couple of ways. I had some background task mules which mostly just ran in a loop with a `sleep()` call. An example is deleting old records from a database once a day. Another example is listening to an Amazon SQS queue for files being uploaded to an S3 bucket. I also had some mules which were triggered by web requests. These are normally what you would use "farms" for. The web request sends a farm message, and a mule picks it up and acts on it. For example, I used this for sending webhook callbacks in response to certain web requests. You could probably also combine this with uwsgi's async mode. That would be useful if you needed a web request to wait for a long running task to finish before sending a response back. I handled that kind of situation with the aforementioned webhook callbacks instead. A sibling comment has mentioned Legion. I've never used that, so can't comment on whether or not the caching and messaging works together with that.
k_bx 7y ago
How does it work on more than one server? (which is how I hope you deploy all your production apps)
- radus 7y ago
  The need for multiple servers kinda depends on the application, no? I'm interested in the single server story.. many of my apps are internal and used by 50 people at most.
  - k_bx 7y ago
    Yeah, but just in terms of redeploy without a downtime, secureness if new version won’t start properly, and redundancy when one server is down because of an OS error while you were asleep. Not about number of clients at all.

d33 7y ago

Could you elaborate, e.g. link to documentation, subprojects or example code? I'd love to get rid of Celery because of how difficult it is for me to tweak it for good performance.

dzonga 7y ago
you can also use dramatiq
darkerside 7y ago
Would love to see more detail as well because I'm a bit skeptical. From uWSGI documentation, it looks like to coordinate cron across anything more than a single server setup, you'd need to configure a Legion, which means you're then integrating uWSGI's orchestration framework with whatever you're already using (k8, ECS, etc). I like minimalism, but sometimes batteries are included for a reason.
- GlennS 7y ago
  I suspect they are using a single server. I found uwsgi's mules and cache2 very useful in that situation. If someone finds that Redis and Celery are more complication than they need for a given task, then I think they're probably not using an orchestration framework.
GlennS 7y ago
This is what I have used in the past. I think they're very convenient if you are just running one instance of uwsgi and want to share some state quickly without going through a persistent database. If you're worrying about tweaking Celery for performance, then I suspect your uses may be a bit more complex than uwsgi's mules are designed for though. https://uwsgi-docs.readthedocs.io/en/latest/Caching.html https://uwsgi-docs.readthedocs.io/en/latest/Caching.html https://uwsgi-docs.readthedocs.io/en/latest/Mules.html https://uwsgi-docs.readthedocs.io/en/latest/Mules.html The cache is a simple key-value store. Works well. I later swapped it out for Redis, because I needed to shared my cache between multiple machines. Switching to Redis was a very quick and easy replacement, so you don't need to worry about lock in. I used mules in a couple of ways. I had some background task mules which mostly just ran in a loop with a `sleep()` call. An example is deleting old records from a database once a day. Another example is listening to an Amazon SQS queue for files being uploaded to an S3 bucket. I also had some mules which were triggered by web requests. These are normally what you would use "farms" for. The web request sends a farm message, and a mule picks it up and acts on it. For example, I used this for sending webhook callbacks in response to certain web requests. You could probably also combine this with uwsgi's async mode. That would be useful if you needed a web request to wait for a long running task to finish before sending a response back. I handled that kind of situation with the aforementioned webhook callbacks instead. A sibling comment has mentioned Legion. I've never used that, so can't comment on whether or not the caching and messaging works together with that.

k_bx 7y ago

How does it work on more than one server? (which is how I hope you deploy all your production apps)

radus 7y ago
The need for multiple servers kinda depends on the application, no? I'm interested in the single server story.. many of my apps are internal and used by 50 people at most.
- k_bx 7y ago
  Yeah, but just in terms of redeploy without a downtime, secureness if new version won’t start properly, and redundancy when one server is down because of an OS error while you were asleep. Not about number of clients at all.