2 ms·

Ask HN: I'm in the market for a distributed message queue, for scheduling background tasks - Does anything support "regional" priorities, where jobs are popped

by mappu 12y ago

Ask HN: I'm in the market for a distributed message queue, for scheduling background tasks - Does anything support "regional" priorities, where jobs are popped based on a combination of age + geographic/latency factors? Also, what are recommended solutions for distributing job injection? My job injection is basically solely dependent on time, and so i envisage one node (raft consensus?) determining all jobs to inject into the queue. My queue volume is about 50 items/sec and nodes will be up to 400ms apart.

Rapzid 12y ago
That sounds like a job for a scheduler, in the cluster sense. I'm actually looking around for a job framework in .NET that supports custom schedulers, but have yet to find something that supports resource constrained scheduling. It's all either about en-queueing something to be done NOW, or at a future date. I haven't seen anything that supports custom scheduler implementations on a per-job type basis. They don't really distinguish between logging work to be done and deciding whether or not it can be executed NOW.