4 ms·
> ~2TB Postgres database into a 70GB Clickhouse was your PG DB been compressed?
by riku_iki 6y ago
> ~2TB Postgres database into a 70GB Clickhouse
was your PG DB been compressed?
- wiredfool 6y agoParts may have been , but it was ~2B rows at ~ 1K/row in PG, with a couple of embedded json fields (which were probably compressed, but ... that's coming from a bad starting point anyway). It got converted to a table with ~25 Float32 columns + a couple of key columns.
- ddorian43 6y agoYes it's an 5-apples vs 1-orange comparison, especially the dynamic json fields where keys are repeated.
- wiredfool 6y agoFixing the PG schema would have saved me about 2/3 of the space. (from a small test run) Good, but not enough to move the needle. Queries on the PG setup here are back to the bad old days of being limited by bulk disk performance.
- riku_iki 6y ago> Good, but not enough to move the needle You could reduce the gap by running PG on compressed filesystem, so it would be more apples to apples comparison.
- ddorian43 6y agoIt would be 1-apple to 1-orange comparison. If the columnar store has an "ok" architecture, it should provide 10x+ faster queries.
- FridgeSeal 6y agoIn addition to what OP mentions in the sibling comment, ClickHouse supports a whole bunch of encodings that can take advantage of column based layouts to further compress the columns in very clever ways