5 ms·

I'm a bit confused by the story. Why did you disagree?

• mrkeen 2y ago
  They asked how I would guarantee that a Postgres update and a Kafka event could both happen transactionally. (P, K) Which sounds like one of those classical impossibility proofs. Their solution was to introduce another part into the system, "the outbox": (P, O) K P and O can form a transaction, but that still leaves the original problem unanswered, how to include K in the transaction.
  • ramchip 2y ago
    The goal of the outbox pattern is at-least-once publishing though, not only-once. You either get P + (eventually) at least one copy of K, or you get no P and no K. Without the outbox you can get P without K or K without P, which lead to consumers out of sync with the producer. This requires the consumer to be able to deal with repeated events to some extent, but you usually want that anyway, since an event can be processed twice even if it appears only once in the topic. For instance if the consumer crashes after processing but before updating the offset.
    • mrkeen 2y ago
      > The goal of the outbox pattern is at-least-once publishing though, not only-once. Right, which is why it's an unacceptable solution to 'transacting over postgres and Kafka', and why I wouldn't want to work for a company that wants me to believe differently. And there's a better solution anyway: just K.
      • ramchip 2y ago
        I think you're just using "transaction" in a different sense than what the interviewer meant; "guarantee that an event happened in addition to a database update" sounds like at-least-once to me, and it's normally what you would want in this kind of system.