3 ms·
This is interesting because Google just created AlloyDb[0] which is decidedly multiprocess for performance and switches out the storage layer from a read/write
by benlivengood 3y ago
This is interesting because Google just created AlloyDb[0] which is decidedly multiprocess for performance and switches out the storage layer from a read/write model to write+replicate + read-only model.
The deep dive[1] has some details; the tl;dr: is that the main process only has to output Write Ahead Logs to the durable storage layer which minimizes transaction latency. The log processing service materializes postgres-compatible on-disk blocks that read-only replicas can read from, with a caching layer for block reads which sends cache invalidations from the LPS to read replicas.
I'm not sure if similar benefits could be seen within a single machine; using network DMA or even rDMA to transfer bytes to and from remote machines also avoids TLB invalidation. There are some mentions in the mailing list of waiting for Linux to support shared page mappings between processes as a solution.
I'm not exactly sure I understand the reasoning behind process separation as crash recovery; as far as I understand each connection is responsible for correctness and so if a process crashes there seems to be an assumption that the database can recover and keep working by killing that process but that seems like it risks silent data corruption; perhaps it's equivalently mitigated by separating materialization of blocks from the sync'd WAL in a separate process from the multithreaded connection process producing WAL entries?
[0] https://cloud.google.com/alloydb https://cloud.google.com/alloydb
[1] https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage https://cloud.google.com/blog/products/databases/alloydb-for...