4 ms·
Parts may have been , but it was ~2B rows at ~ 1K/row in PG, with a couple of embedded json fields (which were probably compressed, but ... that's coming from a
by wiredfool 6y ago
Parts may have been , but it was ~2B rows at ~ 1K/row in PG, with a couple of embedded json fields (which were probably compressed, but ... that's coming from a bad starting point anyway).
It got converted to a table with ~25 Float32 columns + a couple of key columns.
- ddorian43 6y agoYes it's an 5-apples vs 1-orange comparison, especially the dynamic json fields where keys are repeated.
- wiredfool 6y agoFixing the PG schema would have saved me about 2/3 of the space. (from a small test run) Good, but not enough to move the needle. Queries on the PG setup here are back to the bad old days of being limited by bulk disk performance.
- riku_iki 6y ago> Good, but not enough to move the needle You could reduce the gap by running PG on compressed filesystem, so it would be more apples to apples comparison.
- ddorian43 6y agoIt would be 1-apple to 1-orange comparison. If the columnar store has an "ok" architecture, it should provide 10x+ faster queries.