4 ms·
You're right that this is less significant if you are hitting an empty queue (triggering a timeout) between each request. However, even when not comparing (sec
by benmanns 13y ago
You're right that this is less significant if you are hitting an empty queue (triggering a timeout) between each request.
However, even when not comparing (seconds to computer 10000 jobs) benchmarks, you still get benefits from goworker. Since dequeuing, deserialization, and updating stats is faster, the latency between "job insert" and "job started" is lower, which means faster interface updates.
You can also restructure your workers. Instead of running one worker that loops 5,000 times because the Redis overhead is too high, you can have one job queue 5,000 more which can be run on several distributed workers.
- jrochkind1 13y agoYour conclusions about latency, and being able to restructure things so tasks are much smaller -- make sense. (Potentially; I'd still want to benchmark my own real world app before assuming!) (I am confused with your first paragraph and a half, I think you have some typos in there, or I just don't understand what you're saying. But I understand what you're saying about latency and about being able to restructure tasks to be smaller, thanks.)
- benmanns 13y agoAck, sorry for the confusion. I mean that if your users are waiting on jobs, they get the result when `encode time` + `enqueue time` + `wait time` + `dequeue time` + `decode time` + `execute time` is completed. Even though you might not need raw 1000 jobs/second throughput, your users will still benefit by reducing `wait time` + `dequeue time` + `decode time`, which goworker does.