4 ms·
Hard to say for sure without knowing more but I would probably build my own custom protocol for syncing this, because the replication (both binary and logical)
by jeltz 8y ago
Hard to say for sure without knowing more but I would probably build my own custom protocol for syncing this, because the replication (both binary and logical) is built with the assumption that you have a reasonably constant connection from the master to the replica. If not you will start to build up large amounts of write-ahead logs.
- Cthulhu_ 8y agoNever invent the wheel if you haven't done your research yet. Colleagues of mine successfully employed CouchDB in a situation where devices were offline for an x amount of time (running on tablets in airplanes), once they had internet again they would reliably start syncing data with the main database again. This was a number of years ago though, I haven't heard anything about CouchDB since.
- megous 8y agoIt works perfectly with intermittent connectivity, restarts, etc. That will not be a problem. I'm not sure about thousands of nodes syncing to a single replica though. That may be a bit over the top.
- pgaddict 8y agoWho says you need to feed it directly to the replica. Logical replication provides infrastructure for decoding changes, and you may fetch it any way you want / how often you want / feed it wherever you want (file, another database, ...). It would be trivial to write something that connects to a bunch of machines regularly, fetches the decoded increment and feed it somewhere (say, to a single database over a shared connection). See test_decoding / pg_recvlogical for examples.
- pgaddict 8y agoYou need to accumulate the data somewhere. If you don't fetch if from the node, it has to accumulate there. Custom sync protocol does not eliminate this.