4 ms·
-Ofair disables task prefetching which will affect throughput negatively if most of your tasks run for a short time (http://celery.readthedocs.org/en/latest/use
by simonpantzare 11y ago
-Ofair disables task prefetching which will affect throughput negatively if most of your tasks run for a short time (http://celery.readthedocs.org/en/latest/userguide/optimizing.html#prefetch-limits http://celery.readthedocs.org/en/latest/userguide/optimizing...).
- mlissner 11y agoyeah, I don't get this suggestion at all. They say that: > This option comes with a coordination penalty, but results in a much more predictable behavior In general, I'd much rather the greater throughput over predictable behavior. In general, I'm not watching my queue and having it be predictable isn't a very high priority.
- taylorhughes 11y agoOfair will increase throughput in a queue where you have mixed-length tasks, because in a default configuration long tasks will block the queue from starting new tasks. I believe this is because the main process distributes some number of tasks, waits for them to finish, then distributes another set of tasks, and so on. (Would love more clarity on this — I haven't dug into it too much.) Assuming most Celery task queue work involves network requests that can be unpredictable, I think Ofair is a more sensible/understandable default.