4 ms·

If both nodes approve an update on the same primary key, what happens? I don't see this crucial detail described in the README

pgedge_postgres 1y ago
Thanks for pointing out the lack of info on conflict resolution in the README! It's been reported and we'll look at getting that updated ASAP. In the meantime, you can find a lot of information in the official FAQ on how conflict resolution is handled (https://www.pgedge.com/resources/faq https://www.pgedge.com/resources/faq), but at-a-glance, "pgEdge offers eventual consistency between nodes using a configurable policy (e.g. last-writer-wins) for conflict resolution, along with conflict-free delta apply columns (i.e. CRDTs) for running sum fields. This allows for independent, concurrent and eventually consistent updates across multiple nodes."
- n_u 1y ago
  Cool project! How do you generate the timestamps for last writer wins? What happens if there is a tie? Just my 2c: if I see a distributed database, the first question I ask is how it handles distributed transactions. Perhaps this topic should be higher on your FAQ, currently it is the 21st question.
  - bonesmoses 1y ago
    There's an option in the Postgres configuration named "track_commit_timestamp" that does this automatically. It's required to be enabled when using LWW as the conflict resolution model. If there's a tie, the node with the highest node number wins.
pgedge_postgres 1y ago
As a note, there's also specific documentation regarding this: https://docs.pgedge.com/spock_ext/conflicts https://docs.pgedge.com/spock_ext/conflicts And, one of our solutions engineers (Paul Rothrock) has a video released a month ago on this topic as well: https://www.youtube.com/watch?v=prkMkG0SOJE https://www.youtube.com/watch?v=prkMkG0SOJE Sharing these alongside my other comment in case additional information is helpful :-)
bonesmoses 1y ago
In Postgres, updates contain the entire row, including all column values. Since the Spock extension follows the "Last Write Wins" model by default, one row version will win, while the other is essentially discarded. This is assuming the update happened on each node _before_ the new value was synchronized over, or essentially simultaneously. You can address this partially using a CRDT such as the Delta Apply functionality for certain columns: https://docs.pgedge.com/spock_ext/conflicts https://docs.pgedge.com/spock_ext/conflicts That will only work with numeric-type (INT, BIGINT, NUMERIC, etc.) columns, but effectively merges data so updates work cumulatively.

pgedge_postgres 1y ago

Thanks for pointing out the lack of info on conflict resolution in the README! It's been reported and we'll look at getting that updated ASAP. In the meantime, you can find a lot of information in the official FAQ on how conflict resolution is handled (https://www.pgedge.com/resources/faq https://www.pgedge.com/resources/faq), but at-a-glance, "pgEdge offers eventual consistency between nodes using a configurable policy (e.g. last-writer-wins) for conflict resolution, along with conflict-free delta apply columns (i.e. CRDTs) for running sum fields. This allows for independent, concurrent and eventually consistent updates across multiple nodes."

n_u 1y ago
Cool project! How do you generate the timestamps for last writer wins? What happens if there is a tie? Just my 2c: if I see a distributed database, the first question I ask is how it handles distributed transactions. Perhaps this topic should be higher on your FAQ, currently it is the 21st question.
- bonesmoses 1y ago
  There's an option in the Postgres configuration named "track_commit_timestamp" that does this automatically. It's required to be enabled when using LWW as the conflict resolution model. If there's a tie, the node with the highest node number wins.

pgedge_postgres 1y ago

As a note, there's also specific documentation regarding this: https://docs.pgedge.com/spock_ext/conflicts https://docs.pgedge.com/spock_ext/conflicts And, one of our solutions engineers (Paul Rothrock) has a video released a month ago on this topic as well: https://www.youtube.com/watch?v=prkMkG0SOJE https://www.youtube.com/watch?v=prkMkG0SOJE Sharing these alongside my other comment in case additional information is helpful :-)

bonesmoses 1y ago

In Postgres, updates contain the entire row, including all column values. Since the Spock extension follows the "Last Write Wins" model by default, one row version will win, while the other is essentially discarded. This is assuming the update happened on each node _before_ the new value was synchronized over, or essentially simultaneously. You can address this partially using a CRDT such as the Delta Apply functionality for certain columns: https://docs.pgedge.com/spock_ext/conflicts https://docs.pgedge.com/spock_ext/conflicts That will only work with numeric-type (INT, BIGINT, NUMERIC, etc.) columns, but effectively merges data so updates work cumulatively.