4 ms·
Postgres PubSub is great for triggering job dispatch, but it isn’t good enough on its own. There are a few caveats that make polling an additional requirement:
by pselbert 7y ago
Postgres PubSub is great for triggering job dispatch, but it isn’t good enough on its own. There are a few caveats that make polling an additional requirement:
1. Message compaction — notifications are deduplicated, so if two jobs are inserted in the same queue the system may only get one notification.
2. Message saturation — with high levels of activity you’ll need to start discarding messages, essentially denouncing, otherwise the database gets throttled.
3. Dedicated connections — pubsub requires a dedicated connection for listening, which requires a single connection with custom dispatching or one connection per queue.
Relying on PG for everything is awesome regardless!
- StavrosK 7y agoIs this a problem for low-traffic scenarios? My thinking is basically that I should save the extra dependency until I have more than a few tasks per second, at which point I will switch to something like RabbitMQ.
- pselbert 7y agoI don’t think it is much of a problem until you are pushing 2-3k jobs a second. Only the message saturation is a bottleneck anyhow, everything else I mentioned is handled by how a library is architected.
- StavrosK 7y agoThat makes sense, thanks. I generally switch long before I get anywhere near to where the database is the bottleneck, I use Postgres as a broker only for low-traffic sideprojects, where it's much more worthwhile to not bother with an extra broker.