4 ms·

Patroni's documentation also seems to suggest that even with the strongest settings, it can lose transactions; Stolon makes stronger claims.

by aphyr 6y ago

feike 6y ago
Patroni does have synchronous_mode_strict setting, which may be what you're looking for: This parameter prevents Patroni from switching off the synchronous replication on the primary when no synchronous standby candidates are available. As a downside, the primary is not be available for writes (unless the Postgres transaction explicitly turns of synchronous_mode), blocking all client write requests until at least one synchronous replica comes up. https://patroni.readthedocs.io/en/latest/replication_modes.html#replication-modes https://patroni.readthedocs.io/en/latest/replication_modes.h... edit: seems I missed this discussion on twitter: https://twitter.com/jepsen_io/status/1265626035380346881 https://twitter.com/jepsen_io/status/1265626035380346881
- aphyr 6y ago
  Er, again, the docs say "it is still possible to lose transactions even when using synchronous_mode_strict". I've talked about this with some of the Stolon folks on Twitter, and we're not exactly sure how that manifests--possibly an SI or SSI violation.
  - merb 6y ago
    why is stolon more safe than? I tought stolon uses synchronous replication of pg aswell?
satyanash 6y ago
> documentation also seems to suggest that even with the strongest settings, it can lose transactions; Can be reproduced even on a single node postgres. Just hammer it with inserts and maintain a local counter for inserts performed. Then, kill9 the postgres process. You'd expect your local counter to match the actual rows inserted, but you'll find that your counter will always be "less" than the actual rows inserted. Like any "networked" system, it is possible to lose commit acknowledgments even if the commit itself was successful. So yes, you've not "lost" transactions per se. You've "gained" them, but it is still a data issue in either case.
- anarazel 6y ago
  The classical solution to that is to use 2PC. But often it's not worth it...
  - andreareina 6y ago
    2PC can have unacknowledged writes: https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres
qeternity 6y ago
Ah, I presumed you were talking about distributed failure situations (split brain, etc) as opposed the to PG level replication (which most solutions orchestrate anyway).

Patroni's documentation also seems to suggest that even with the strongest settings, it can lose transactions; Stolon makes stronger claims.

by aphyr 6y ago

Patroni's documentation also seems to suggest that even with the strongest settings, it can lose transactions; Stolon makes stronger claims.

feike 6y ago

Patroni does have synchronous_mode_strict setting, which may be what you're looking for: This parameter prevents Patroni from switching off the synchronous replication on the primary when no synchronous standby candidates are available. As a downside, the primary is not be available for writes (unless the Postgres transaction explicitly turns of synchronous_mode), blocking all client write requests until at least one synchronous replica comes up. https://patroni.readthedocs.io/en/latest/replication_modes.html#replication-modes https://patroni.readthedocs.io/en/latest/replication_modes.h... edit: seems I missed this discussion on twitter: https://twitter.com/jepsen_io/status/1265626035380346881 https://twitter.com/jepsen_io/status/1265626035380346881

aphyr 6y ago
Er, again, the docs say "it is still possible to lose transactions even when using synchronous_mode_strict". I've talked about this with some of the Stolon folks on Twitter, and we're not exactly sure how that manifests--possibly an SI or SSI violation.
- merb 6y ago
  why is stolon more safe than? I tought stolon uses synchronous replication of pg aswell?

satyanash 6y ago

> documentation also seems to suggest that even with the strongest settings, it can lose transactions; Can be reproduced even on a single node postgres. Just hammer it with inserts and maintain a local counter for inserts performed. Then, kill9 the postgres process. You'd expect your local counter to match the actual rows inserted, but you'll find that your counter will always be "less" than the actual rows inserted. Like any "networked" system, it is possible to lose commit acknowledgments even if the commit itself was successful. So yes, you've not "lost" transactions per se. You've "gained" them, but it is still a data issue in either case.

anarazel 6y ago
The classical solution to that is to use 2PC. But often it's not worth it...
- andreareina 6y ago
  2PC can have unacknowledged writes: https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres

qeternity 6y ago

Ah, I presumed you were talking about distributed failure situations (split brain, etc) as opposed the to PG level replication (which most solutions orchestrate anyway).