3 ms·

Yeah, but even without tuning, you can have hundreds of "sparse" queues with this pattern - you don't have to care about the frequency of insertions or deletion

Yeah, but even without tuning, you can have hundreds of "sparse" queues with this pattern - you don't have to care about the frequency of insertions or deletions or the size of the rows.

gigatexal 7y ago
I mean I agree I think with what I think you’re saying: as much as I love Postgres and SQL I’d much rather use Redis for this.
- colinchartier 7y ago
  I actually used redis pub/sub before switching to triggers - it is significantly easier to just: INSERT INTO JOBS versus INSERT INTO JOBS redis-cli LPUSH job-queue job-id It's also easier to configure, redis has BRPOPLPUSH for acked-queues, but it's much easier to just query jobs with a select statement from the database.
- Rapzid 7y ago
  Obviously "it depends" but here are a few reasons one may want to use PostgreSQL instead: 1.) *They want to transactionally commit work along with the change that caused it 2.) They are already using Postgresql not using Redis 3.) Requiring users install yet another service(Redis) just for this one item isn't worth the costs
  - gigatexal 7y ago
    Redis modifications are transactional: https://redis.io/topics/transactions https://redis.io/topics/transactions Though I agree on points 2 and 3, especially 3 since it adds complexity.
    - Rapzid 7y ago
      Internally but if the change causing the work is in PostgreSQL Redis would need to support 2-phase commits to commit the change and the work together atomically(and durably).
    - erichocean 7y ago
      > Redis modifications are transactional That's only true if they do not error—there is no "rollback" feature in Redis, and if you do error, anything you've done up to that point remains. That's not transactional, it's more "you can, sometimes, execute a bunch of separate statement atomically".
      - gigatexal 7y ago
        I’ll admit I don’t know the internals of Redis as well as I do relational databases. I’ve learned much from this thread.

Yeah, but even without tuning, you can have hundreds of "sparse" queues with this pattern - you don't have to care about the frequency of insertions or deletions or the size of the rows.

gigatexal 7y ago

I mean I agree I think with what I think you’re saying: as much as I love Postgres and SQL I’d much rather use Redis for this.

colinchartier 7y ago
I actually used redis pub/sub before switching to triggers - it is significantly easier to just: INSERT INTO JOBS versus INSERT INTO JOBS redis-cli LPUSH job-queue job-id It's also easier to configure, redis has BRPOPLPUSH for acked-queues, but it's much easier to just query jobs with a select statement from the database.
Rapzid 7y ago
Obviously "it depends" but here are a few reasons one may want to use PostgreSQL instead: 1.) *They want to transactionally commit work along with the change that caused it 2.) They are already using Postgresql not using Redis 3.) Requiring users install yet another service(Redis) just for this one item isn't worth the costs
- gigatexal 7y ago
  Redis modifications are transactional: https://redis.io/topics/transactions https://redis.io/topics/transactions Though I agree on points 2 and 3, especially 3 since it adds complexity.
  - Rapzid 7y ago
    Internally but if the change causing the work is in PostgreSQL Redis would need to support 2-phase commits to commit the change and the work together atomically(and durably).
  - erichocean 7y ago
    > Redis modifications are transactional That's only true if they do not error—there is no "rollback" feature in Redis, and if you do error, anything you've done up to that point remains. That's not transactional, it's more "you can, sometimes, execute a bunch of separate statement atomically".
    - gigatexal 7y ago
      I’ll admit I don’t know the internals of Redis as well as I do relational databases. I’ve learned much from this thread.