4 ms·

Read a while back about a new storage backend named zheap that was meant to obviate the need for Vaccuum. Was from a year or two ago and at the time they were

by tux1968 7y ago

Read a while back about a new storage backend named zheap that was meant to obviate the need for Vaccuum. Was from a year or two ago and at the time they were aiming for PG 12. Wonder where they stand now. Looking at their Github, it's still active: https://github.com/EnterpriseDB/zheap https://github.com/EnterpriseDB/zheap

Tostino 7y ago
They were the driving force getting the pluggable storage API into pg12. zheap is planning on using that API.
anarazel 7y ago
The pluggable table storage abstraction in v12 is one major stepping stone towards zheap. Another major piece (general undo infrastructure), is being discussed on the PG development list right now.
atombender 7y ago
Zheap's design [1] sounds almost exactly like Oracle. The way Oracle works, when you do transactional row update (e.g. UPDATE), the previous version is read and copied to the undo log, and then the update is performed in-place in the table even though it's not committed yet. Each row (or maybe it's per page — it's been a while) has a "transaction slots" header, and the transaction then writes its ID to this list. If another, concurrent transaction does a SELECT against the same row, it will see in the slot list that this row is from an uncommitted transaction, and if your transaction isolation level is high enough, it will then look into the undo log for the currently committed row. This design means that there's no transactional overhead for reads done against rows that aren't being written to, so it's optimized for the general case — performance is only impacted when a transaction has dirty rows and other transactions have to dip into the undo log to find the old versions. The downside is that commits and rollbacks are slower; for commits, all the transaction slots need to be updated, and for rollbacks the undo log needs to be copied back into the table. Postgres's current MVCC is very elegant in theory — MVCC means never updating anything in-place, always appending, so there's never any read/write contention, and commits/rollbacks are near-instantaneous. But the garbage collection aspect ("vacuum") has always been a challenge, and Postgres has mitigated a lot of performance issues over time with clever hacks that compromise the purity of the MVCC model (e.g., HOT tuples, index visibility bits). I think it's definitely time for alternative approaches. [1] https://github.com/EnterpriseDB/zheap/blob/master/src/backend/access/zheap/README https://github.com/EnterpriseDB/zheap/blob/master/src/backen...
- iracic 7y ago
  >> The downside is that commits and rollbacks are slower Commit is not slower. Some part of the work is delayed, ex first SELECT after change may be involved to check & update row status in data block by peeking transaction state in UNDO. Huge rollback is the real pain and something you don't want to do and in some situations you would prefer to go UNDO-less.

Tostino 7y ago

They were the driving force getting the pluggable storage API into pg12. zheap is planning on using that API.

anarazel 7y ago

The pluggable table storage abstraction in v12 is one major stepping stone towards zheap. Another major piece (general undo infrastructure), is being discussed on the PG development list right now.

atombender 7y ago

Zheap's design [1] sounds almost exactly like Oracle. The way Oracle works, when you do transactional row update (e.g. UPDATE), the previous version is read and copied to the undo log, and then the update is performed in-place in the table even though it's not committed yet. Each row (or maybe it's per page — it's been a while) has a "transaction slots" header, and the transaction then writes its ID to this list. If another, concurrent transaction does a SELECT against the same row, it will see in the slot list that this row is from an uncommitted transaction, and if your transaction isolation level is high enough, it will then look into the undo log for the currently committed row. This design means that there's no transactional overhead for reads done against rows that aren't being written to, so it's optimized for the general case — performance is only impacted when a transaction has dirty rows and other transactions have to dip into the undo log to find the old versions. The downside is that commits and rollbacks are slower; for commits, all the transaction slots need to be updated, and for rollbacks the undo log needs to be copied back into the table. Postgres's current MVCC is very elegant in theory — MVCC means never updating anything in-place, always appending, so there's never any read/write contention, and commits/rollbacks are near-instantaneous. But the garbage collection aspect ("vacuum") has always been a challenge, and Postgres has mitigated a lot of performance issues over time with clever hacks that compromise the purity of the MVCC model (e.g., HOT tuples, index visibility bits). I think it's definitely time for alternative approaches. [1] https://github.com/EnterpriseDB/zheap/blob/master/src/backend/access/zheap/README https://github.com/EnterpriseDB/zheap/blob/master/src/backen...

iracic 7y ago
>> The downside is that commits and rollbacks are slower Commit is not slower. Some part of the work is delayed, ex first SELECT after change may be involved to check & update row status in data block by peeking transaction state in UNDO. Huge rollback is the real pain and something you don't want to do and in some situations you would prefer to go UNDO-less.