4 ms·
I'm not familiar with Postgres (or any database, for that matter) internals: how does it store variable-length types? I'm assuming that it doesn't do the tradit
by lights0123 6y ago
I'm not familiar with Postgres (or any database, for that matter) internals: how does it store variable-length types? I'm assuming that it doesn't do the traditional pointer-into-the-heap, and maybe does more like this:
struct Data { size_t len; uint8_t data[] };
Does it store the length in some metadata section, or inline like that struct (I'm assuming not, as then lookups would be expensive with having to follow basically a linked list)? How does it handle resizes?
- anarazel 6y agoFor on-disk storage (in memory largely is different, although the on-disk format is used in a few places too), variable length datums are stored as 'varlenas'. Basically there's a 'varlena header' indicating the length of in-line stored data (1 byte for short contents up to 126 bytes long, 4 byte header after that). For data longer than 126 bytes the data can be compressed inline. Besides being storing the actual data, compressed or uncompressed, inline, the inline data may also just be a pointer to data stored outside the main table ("TOAST"). Where data again can be compressed or uncompressed depending on settings and compressability. We do utilize pointers-into-the-buffer-pool in some places, to point to variable length columns in the buffer pool (obviously only while we have pinned the respective page(s)). That's never used for by-value data types. Hopes that answers your question?
- anarazel 6y agoProbably not helpful, and obviously one of the more archaic parts of postgres, but here's the concrete encoding: https://github.com/postgres/postgres/blob/master/src/include/postgres.h#L168 https://github.com/postgres/postgres/blob/master/src/include...