4 ms·
>>>> 2. Postgres preserves your data if it goes down >>>Not if you're using a SKIP LOCKED queue, Can you provide a reference for this? I'm not aware that usi
by andrewstuart 7y ago
>>>> 2. Postgres preserves your data if it goes down
>>>Not if you're using a SKIP LOCKED queue,
Can you provide a reference for this? I'm not aware that using SKIP LOCKED data is treated any differently to other Postgres data in terms of durability.
- KirinDave 7y agoIt's exactly the same case as SQS in that if your disk interface dies in mid write of a message you lose it the same as if your interface to SQS or SQS itself dies mid-write, you lose it. The rest of the sentence is a challenge, "show me a data loss bug in SQS that extends beyond the failure cases you've already implicitly agreed to using postgres itself."
- gmueckl 7y agoWhat possibilities are there for a write to fail that the database server cannot react to? I am under the impression that a write error like this would be reported as a failure to run the statement and the application is responsible for handling that.
- KirinDave 7y agoThe box turning off, the disk interface failing, and/or the link to the database failing in mid instruction. Same as SQS.
- xyzzy_plugh 7y agoAll of those would result in a write failure from the application's perspective, which is fine, and must be accounted for regardless (e.g. retry, two phase commit, log an error, whatever).
- z3t4 7y agoAlso the FS might report the data as written but its actually in a write cache and will be lost if the plug is pulled.
- gmueckl 7y agoNot an issue if you follow th e recommendations in the PostgeSQL documentation. The way to comfigure write caches is described there.
- mcqueenjordan 7y agoBut you have to explicitly delete the message from SQS, right? You'd only delete after confirming you processed the message, right? So if you die mid-instruction in processing a message, the message just re-appears in the SQS queue after the visibility timeout.
- KirinDave 7y agoThat's a feature, and one your have to replicate in your postgres queue.
- cryptonector 7y agoSKIP LOCKED is a read-side feature. PG is a problem for u/andrewstuart because they have to manage it themselves (unless they're getting a PG service from a cloudy provider, say) and that's harder than it is for Amazon to manage SQS.