3 ms·
If you wanted to 'publish' a more interactive website to the internet while still keeping all of the publishing tools safe inside an enclave/intranet somewhere,
by hinkley 5y ago
If you wanted to 'publish' a more interactive website to the internet while still keeping all of the publishing tools safe inside an enclave/intranet somewhere, then you probably need a read-only copy of a database. But any time I look at read-replica configurations, it looks like they expect the replica to contact the master. You can't set up a proper bastion server if you have to have a secret door back into your intranet to replicate data. This has left me stuck on the starting blocks for a couple of pet projects I might like to work on.
Anybody know how to solve this problem, short of roll-your-own?
- int0x2e 5y agoHow much latency and inconsistency can you live with? Could you dump full or incremental database backups from your secure enclave, upload them to blobs and ingest on the cloud? Could you publish a change feed of sorts to Kafka and apply changes?
- hinkley 5y agoIt would have to be incremental because a full dump does not scale up. Of course if you scale down you could probably get away with shipping snapshots of a sqlite database, and I understand some people do just that. But it would be nice to have streaming WAL logs with the instigator flipped. Probably I need some sort of middleware to dis-intermediate the pipe.
- teraflop 5y agoWith Postgres, you can do log-shipping replication using whatever mechanism you like to deliver the WAL files, with no direct network connection between the primary and standby servers. It's still kind of roll-your-own, because you need to provide shell scripts to handle archiving, fetching and cleaning up the log files on either end, but it can be done.
- ivank 5y agoWireGuard