4 ms·

I had a similar question, given that I'm happily processing jobs in Elixir that are enqueued by a Ruby app via Sidekiq. After taking a look at the README's, it

I had a similar question, given that I'm happily processing jobs in Elixir that are enqueued by a Ruby app via Sidekiq. After taking a look at the README's, it looks like what this gets you is dumb clients, with the logic around retries, etc built into the server, rather than forcing the client to handle these details. That's great for polyglot systems—the dumber the client the better, in my book. As another commenter posted, I'll be interested to see more about what tradeoffs it makes in distribution. If your current system is already something like SNS/SQS or an actual message queue with acks, then this probably isn't aimed at you.

manigandham 9y ago
Why run something extra when you can use a single database table which is already universally accessible by any dumb client? Push, fetch, ack = insert, select, delete. A few lines of SQL or a stored procedure gets it done. Postgres basically made skip locked for queues.
- karmajunkie 9y ago
  Its a valid way to go, but does skip locked handle retries for you? There's more to a queue than just the message. Also worth noting, postgres is also another piece of infrastructure to run, and not all applications use it.
  - manigandham 9y ago
    The whole point is that Faktory is another piece of infrastructure, with arguably little value when you already have all of the features within Redis or a SQL database which you're probably running anyway. Sidekiq itself requires Redis. For postgres, SKIP LOCKED is the easiest way to queue because the row only gets deleted if the transaction commits, which is when the job finishes. Otherwise if there's an error than don't commit or just crash and the row remains in the queue and another worker process tries it again.
    - karmajunkie 9y ago
      With redis you're back to the original problem of needing to code things like retries back into your client. Postgres may have some limited abilities to use it as a message queue but you're still going to be missing some of the other features that will end up in faktory like periodic jobs. If pg works for your situation, that's great, but i don't know why you'd assume that none of these objections have come up before and been responded to.
      - manigandham 9y ago
        If you want the client-side retry logic too then that requires language-specific libraries, and it seems that's what Faktory requires anyway so this whole thing becomes unnecessary. They couldve just ported the sidekiq library to several languages or use a single C library with language wrappers and gotten the same result with less work.

manigandham 9y ago

Why run something extra when you can use a single database table which is already universally accessible by any dumb client? Push, fetch, ack = insert, select, delete. A few lines of SQL or a stored procedure gets it done. Postgres basically made skip locked for queues.

karmajunkie 9y ago
Its a valid way to go, but does skip locked handle retries for you? There's more to a queue than just the message. Also worth noting, postgres is also another piece of infrastructure to run, and not all applications use it.
- manigandham 9y ago
  The whole point is that Faktory is another piece of infrastructure, with arguably little value when you already have all of the features within Redis or a SQL database which you're probably running anyway. Sidekiq itself requires Redis. For postgres, SKIP LOCKED is the easiest way to queue because the row only gets deleted if the transaction commits, which is when the job finishes. Otherwise if there's an error than don't commit or just crash and the row remains in the queue and another worker process tries it again.
  - karmajunkie 9y ago
    With redis you're back to the original problem of needing to code things like retries back into your client. Postgres may have some limited abilities to use it as a message queue but you're still going to be missing some of the other features that will end up in faktory like periodic jobs. If pg works for your situation, that's great, but i don't know why you'd assume that none of these objections have come up before and been responded to.
    - manigandham 9y ago
      If you want the client-side retry logic too then that requires language-specific libraries, and it seems that's what Faktory requires anyway so this whole thing becomes unnecessary. They couldve just ported the sidekiq library to several languages or use a single C library with language wrappers and gotten the same result with less work.