2 ms·

Hey, about the column-wide data compression thing. If I have a column with strings that are highly similar to each other (maybe they are urls, maybe they are HT

Hey, about the column-wide data compression thing. If I have a column with strings that are highly similar to each other (maybe they are urls, maybe they are HTML documents that were scraped from the same site, but they are expected to contain redundancies) What's the best approach to get excellent compression? Can I exploit this redundancy within the column somehow? I don't even know the right search terms for this. TimescaleDB compression will work only for integer columns, right? And in any case, this is not a time series.

mfreed 4y ago
TimescaleDB supports compression for all data types, it just uses type-aware compression algorithms that it applies automatically/transparently based on typing. - Gorilla compression for floats - Delta-of-delta + Simple-8b with run-length encoding compression for timestamps and other integer-like types - Whole-row dictionary compression for columns with a few repeating values (+ LZ compression on top) - LZ-based array compression for all other types https://www.timescale.com/blog/building-columnar-compression-in-a-row-oriented-database/ https://www.timescale.com/blog/building-columnar-compression... So as to your question, just turn on compression; it's very common to see 94-97% reduction in storage. (Timescale co-founder)