3 ms·

For local cluster, try DRBD to replicate at the disk level. The DRBD replication can be synchronous and are very fast. Add Linux-HA for health monitoring and

by ww520 9y ago

For local cluster, try DRBD to replicate at the disk level. The DRBD replication can be synchronous and are very fast. Add Linux-HA for health monitoring and automatic failover. When failed over, the standby machine just starts up PostgreSQL and recovers from the transaction log. Uncommitted transactions on the primary machine that haven't been written to disk and replicated will be lost, which is consistent with the transaction semantic. Committed transactions are replicated and recovered from the transaction log at the standby. Depending on the heartbeat interval and how fast PostgreSQL starts up and recovers from the log, the automatic failover can happen within seconds. Configure a virtual IP for the cluster and let the standby machine to take over the IP at failover. Linux-HA does all that for you. The clients talk to the virtual IP and don't have to know the primary or standby machine. They just need to reconnect when disconnected at the failover. The nice thing about DRBD and Linux-HA is that it's not just for PostgreSQL. Any disk based application can be benefited.

takeda 9y ago
I would strongly discourage this kind of solution. DRBD is to replicate disk. As pointed out is not Postgres specific, and it doesn't understand the data it replicates. One of nightmare of replication is that there's latency and data never makes it instantly to the other host. This means that there is a possibility that once the master fails, the database file on the other host might be in a corrupted state. If your data is important you should use replication mechanisms provided by the database. Repmgr[1] makes it easier to set up replication, and perform failover. It also has mechanisms to do auto failover, but I would also be careful with that. [1] http://repmgr.org/ http://repmgr.org/
- ww520 9y ago
  DRBD's replication is synchronous. That means the block device's write() call on the primary won't return until the data is written onto the standby's disk. It's is as strong a guarantee as you can get. In case of the secondary not available, the data are queued up in the primary and sync up with the secondary when it comes back online. If the primary's disk is destroyed at that time, the failure case and data loss is the same as the database's native replication. DRBD is much simpler and less error prone precisely because it does not care to understand the application data. It just deals with disk block. The simplicity also helps performance greatly. Benchmark has shown there's negligible impact when doing synchronous replication.
  - takeda 9y ago
    I guess that's better, but it would just decrease performance. I still think using binlogs is safer and more efficient though, since it transfers only what's needed to make an update.
    - ww520 9y ago
      Performance is the same if not better than database level replication. The database level replication is synchronous too and have to wait for the standby db to finish writing. If it's async, it has a worse data loss problem. The DRBD replication is fast because it intercepts the write call and then issues the network send and writing to the underlying disk in parallel. Gigabit or 10G network IO are magnitudes faster than disk seek/write. The write to disk on the standby server can happen in parallel as the primary's write to underlying disk.

by ww520 9y ago

takeda 9y ago

I would strongly discourage this kind of solution. DRBD is to replicate disk. As pointed out is not Postgres specific, and it doesn't understand the data it replicates. One of nightmare of replication is that there's latency and data never makes it instantly to the other host. This means that there is a possibility that once the master fails, the database file on the other host might be in a corrupted state. If your data is important you should use replication mechanisms provided by the database. Repmgr[1] makes it easier to set up replication, and perform failover. It also has mechanisms to do auto failover, but I would also be careful with that. [1] http://repmgr.org/ http://repmgr.org/

ww520 9y ago
DRBD's replication is synchronous. That means the block device's write() call on the primary won't return until the data is written onto the standby's disk. It's is as strong a guarantee as you can get. In case of the secondary not available, the data are queued up in the primary and sync up with the secondary when it comes back online. If the primary's disk is destroyed at that time, the failure case and data loss is the same as the database's native replication. DRBD is much simpler and less error prone precisely because it does not care to understand the application data. It just deals with disk block. The simplicity also helps performance greatly. Benchmark has shown there's negligible impact when doing synchronous replication.
- takeda 9y ago
  I guess that's better, but it would just decrease performance. I still think using binlogs is safer and more efficient though, since it transfers only what's needed to make an update.
  - ww520 9y ago
    Performance is the same if not better than database level replication. The database level replication is synchronous too and have to wait for the standby db to finish writing. If it's async, it has a worse data loss problem. The DRBD replication is fast because it intercepts the write call and then issues the network send and writing to the underlying disk in parallel. Gigabit or 10G network IO are magnitudes faster than disk seek/write. The write to disk on the standby server can happen in parallel as the primary's write to underlying disk.