4 ms·

How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all

by b777 9y ago

How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? And if all writes need to happen on all servers, how does that improve write scalability as the load write load on each server is the same as it would be without multi-master replication.

anarazel 9y ago
> How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? It doesn't generally. You often can increase the total throughput because independent transactions can be batch committed when replicated, which can increase throughput noticeably, but that's more like a constant factor improvement than a real scalability improvement. What it does however often allow you to do is to: a) Address latency issues with writers distributed across the globe. If you have masters on each continent your total write rate can often be massively higher than if you have application servers from each of those continents write to a centralized database. b) Partially replicate your data globally, partially not. Quite often you'll want to have some data available everywhere (typically metadata, security related data, etc), but a lot of the bulk data only available in certain places.
- darkr 9y ago
  > a) Address latency issues with writers distributed across the globe How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Same with incrementing IDs etc.
  - anarazel 9y ago
    > > a) Address latency issues with writers distributed across the globe > How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Well, there's different systems out there. What you want for something like this is asynchronous multi-master. Which, as you observe, allows for conflicts to occur, which need to be resolved somehow. > Same with incrementing IDs etc. The solution for those is usually to have each node assign sequences with different start values, have a system that distributes "chunks" of values (that was what BDR, an async MM version of postgres used to do), or avoiding sequences in favor of UUIDs or similar.

by b777 9y ago

anarazel 9y ago

> How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? It doesn't generally. You often can increase the total throughput because independent transactions can be batch committed when replicated, which can increase throughput noticeably, but that's more like a constant factor improvement than a real scalability improvement. What it does however often allow you to do is to: a) Address latency issues with writers distributed across the globe. If you have masters on each continent your total write rate can often be massively higher than if you have application servers from each of those continents write to a centralized database. b) Partially replicate your data globally, partially not. Quite often you'll want to have some data available everywhere (typically metadata, security related data, etc), but a lot of the bulk data only available in certain places.

darkr 9y ago
> a) Address latency issues with writers distributed across the globe How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Same with incrementing IDs etc.
- anarazel 9y ago
  > > a) Address latency issues with writers distributed across the globe > How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Well, there's different systems out there. What you want for something like this is asynchronous multi-master. Which, as you observe, allows for conflicts to occur, which need to be resolved somehow. > Same with incrementing IDs etc. The solution for those is usually to have each node assign sequences with different start values, have a system that distributes "chunks" of values (that was what BDR, an async MM version of postgres used to do), or avoiding sequences in favor of UUIDs or similar.