2 ms·
Would it be a good idea to use zstd's dictionary compression for databases? It seems like it could be a big win since you are compressing relatively small block
by adgjlsfhk1 5y ago
Would it be a good idea to use zstd's dictionary compression for databases? It seems like it could be a big win since you are compressing relatively small blocks with known structure.
- felixhandte 5y agoYes. For example, RocksDB uses zstd dictionaries: https://rocksdb.org/blog/2021/05/31/dictionary-compression.html https://rocksdb.org/blog/2021/05/31/dictionary-compression.h...
- infogulch 5y agoI've wondered the same. Maybe a per-table dictionary created as part of CREATE STATISTICS? I guess the difficulty will be updating the dictionary, you might need some kind of page epoch to look up what the dict was at the time the page was written.