4 ms·

I think SQS is cheap enough to build on as a messaging queue even if you're not hosting within AWS. Out of the widely underrated AWS services include SNS and S

• qaq 3y ago
  SQS is at least once PG can give you exactly once
  • qaq 3y ago
    Not sure why this is making people upset.
    • williamdclt 3y ago
      Because it's incorrect. If you have any non-postgres side-effect, you can't have exactly-once (unless you do 2PC or something like that). There isn't any technology that gives you "exactly once" in the general case.
      • qaq 3y ago
        That's not how exactly once is defined for queue. We are talking about semantics of what queue systems is providing.
        • silon42 3y ago
          Nobody will understand it like that.
          • qaq 3y ago
            Anyone who has ever selected queue service/product will understand it like that. Because thats one of the most prominent features that gets highlighted by those products: SQS Standard queues support at-least-once message delivery. NATS offers "at-most-once" delivery etc.
  • rcaught 3y ago
    SQS FIFO has exactly-once processing
    • qaq 3y ago
      well that's a stretch it has "5 minute window" You can hold a lock on a row in PG queue for as long as you need
      • chuckhend 3y ago
        pgmq (which is linked on this gist) provides an api to this functionality. It can be 0 seconds, or 10 years if you want. It's not a row lock in, which can be expensive. In pgmq, its build into the design of the visibility timeout. FOR UPDATE SKIP LOCKED is there to ensure that only a single consumer gets any message, and then the visibility timeout lets consumer determine how long it should continue to remain unavailable to other consumers.
• chuckhend 3y ago
  You get exactly once when you consume with pgmq and run your queue operations inside transactions in your postgres database. I can't think of an easy way to get some equivalent on SQS without building something like an outbox.
  • rcaught 3y ago
    SQS FIFO has exactly-once processing
    • vp8989 3y ago
      That's not what the parent post was referring to. If SQS (or your ability to talk to it) is down and your database isn't, what do you do?
• williamdclt 3y ago
  The problem is rarely cost, it's operational overhead. Using SQS for a queue rather than my already-existing Postgres means that I have to: - Write a whole bunch of IaC, figuring out the correct access policies - Set up monitoring: figure out how to monitor, write some more IaC - Worry about access control: I just increased the attack surface of my application - Wire it up in my application so that I can connect to SQS - Understand how SQS works, how to use its API It's often worth it, but adding an additional moving piece into your infra is always a lot of added cognitive load.
  • fuy 3y ago
    +. And then you have to figure everything one more time when you decide to move to (or to add support for) Azure/GCP.