4 ms·
My gripe with using a relational database as a priority queue is tends not to scale well. (Similarly as if you tried to use a relational database as a queue).
by nemothekid 11y ago
My gripe with using a relational database as a priority queue is tends not to scale well. (Similarly as if you tried to use a relational database as a queue).
Once you get to the 10s of millions of messages per day (for example ~8M at priority X and 2M at priority Y), performance tends to go down the drain (due to contention), disk space bloats (due to vacumming/deletes).
I've yet to come across a good task scheduling system that will take into the account resources available and the priority of the tasks, while still usable on a largish dataset that isn't a pain to maintain.
- manigandham 11y agoParent is only talking about keeping the job state in a database, not the queue messages.
- nemothekid 11y agoHow does that work? If the the queue is ultimately delivering the messages, doesn't that mean you can't have priority based queuing?