3 ms·

> It’s better to think of BDR like other eventually consistent data stores […] with the exception that this will never be consistent. Silently corrupting data

by pilif 11y ago

> It’s better to think of BDR like other eventually consistent data stores […] with the exception that this will never be consistent. Silently corrupting data goes totally against the usual Postgres spirit (which might be one of the reasons why the BDR patches haven't been accepted yet). Normally, if $THING can't be guaranteed to be done right, then Postgres just doesn't offer the feature (or throws a specific error). If handling transactions isn't feasible for BDR, then it should refuse BEGIN's instead of accepting them and silently corrupting data. If handling transactions across tables isn't feasible for BDR, it should blow up once you touch more than one table, forcing you to roll the whole transaction (which has so far only touched one table) back. It's totally fine if your tool can't do $THING. Especially in matters of multi master replication which is a really hard problem. However, I would really want your tool to be honest about its deficiencies and tell me so I can try to find a workaround. (Of course this could also just be a "simple" bug at which point, you can disregard this rant and just fix the issue :p)

saosebastiao 11y ago
Agreed. This is the sort of thing you would expect from a Postgres fork, but not Postgres itself.
anarazel 11y ago
> Silently corrupting data goes totally against the usual Postgres spirit (which might be one of the reasons why the BDR patches haven't been accepted yet). Normally, if $THING can't be guaranteed to be done right, then Postgres just doesn't offer the feature (or throws a specific error). There's a good set of scenarios, particularly globally distributed datastores, where latency makes it infeasible to make strong guarantees. That's what BDR was designed for. That's why you can configure conflict handling and add conflict handlers to resolve conflicts. And it's not "silently corrupting" things - it's logging the conflicts to postgresql's log and, if configured, to a historical table. > If handling transactions isn't feasible for BDR, then it should refuse BEGIN's instead of accepting them and silently corrupting data. That just depends on what you define as "handling transactions". You can have multi statement transactions which are atomic - something rather useful for a lot of usecases. Other sessions, on any node, will never see a partially applied set of DML from within a transaction. Also useful. I agree that it'd be helpful to expand BDR docs on the specifics of what consistency to expect when doing what.
- pilif 11y ago
  >And it's not "silently corrupting" things - it's logging the conflicts to postgresql's log and, if configured, to a historical table. I stand corrected. Still, it would be useful if there was a mode when it blew up because that would make sure the data is always consistent even if the application forgets to check the historical table.
anarazel 11y ago
Oh and: > which might be one of the reasons why the BDR patches haven't been accepted yet That's primarily because they've not been submitted as a whole. Many individual parts of BDR have been submitted & integrated into postgres. Simplified versions of others (the logical decoding output plugin, the apply side, replication set management) have been submitted recently and are being reviewed. I don't think any major part has been wholly rejected - changed during review, surely and heavily.

by pilif 11y ago

saosebastiao 11y ago

Agreed. This is the sort of thing you would expect from a Postgres fork, but not Postgres itself.

anarazel 11y ago

> Silently corrupting data goes totally against the usual Postgres spirit (which might be one of the reasons why the BDR patches haven't been accepted yet). Normally, if $THING can't be guaranteed to be done right, then Postgres just doesn't offer the feature (or throws a specific error). There's a good set of scenarios, particularly globally distributed datastores, where latency makes it infeasible to make strong guarantees. That's what BDR was designed for. That's why you can configure conflict handling and add conflict handlers to resolve conflicts. And it's not "silently corrupting" things - it's logging the conflicts to postgresql's log and, if configured, to a historical table. > If handling transactions isn't feasible for BDR, then it should refuse BEGIN's instead of accepting them and silently corrupting data. That just depends on what you define as "handling transactions". You can have multi statement transactions which are atomic - something rather useful for a lot of usecases. Other sessions, on any node, will never see a partially applied set of DML from within a transaction. Also useful. I agree that it'd be helpful to expand BDR docs on the specifics of what consistency to expect when doing what.

pilif 11y ago
>And it's not "silently corrupting" things - it's logging the conflicts to postgresql's log and, if configured, to a historical table. I stand corrected. Still, it would be useful if there was a mode when it blew up because that would make sure the data is always consistent even if the application forgets to check the historical table.

anarazel 11y ago

Oh and: > which might be one of the reasons why the BDR patches haven't been accepted yet That's primarily because they've not been submitted as a whole. Many individual parts of BDR have been submitted & integrated into postgres. Simplified versions of others (the logical decoding output plugin, the apply side, replication set management) have been submitted recently and are being reviewed. I don't think any major part has been wholly rejected - changed during review, surely and heavily.