2 ms·
I just want a recognized standard format for write ahead logs. Start with replicating data between OLTP and OLAP databases with minimal glue code, and start mov
by hinkley 2y ago
I just want a recognized standard format for write ahead logs. Start with replicating data between OLTP and OLAP databases with minimal glue code, and start moving other systems to a similar structure, like Kafka, then new things we haven’t thought of yet.
- ianburrell 2y agoThe structure for the write head logs needs to different between systems. For Postgres, the WAL is a record of writes with new blocks. It can't be used without knowing the Postgres disk format. I don't think it can be used to construct logical changes. Using a standard format, converting things into logical data, would be significantly slower. It is important that WAL be fast because it is the bottleneck in transactions. It would make more sense to have a separate change streaming service.
- anarazel 2y ago> I don't think it can be used to construct logical changes. It can: https://www.postgresql.org/docs/current/logicaldecoding.html https://www.postgresql.org/docs/current/logicaldecoding.html It's not entirely from the WAL though, some catalog accesses are necessary for metadata (shape and name of tables etc).