3 ms·
To be honest any file format that would do 1) strong typing/binary data, 2) string deduplication would capture most of the advantages of parquet, except being a
by cm2187 4y ago
To be honest any file format that would do 1) strong typing/binary data, 2) string deduplication would capture most of the advantages of parquet, except being a column store format which helps in some cases and hurts in others.
- julik 4y agoAnd even using a columnar format but without all the complexity of Parquet would be comparatively easy (when writing using buffers)
- nomadluap 4y agoAt that point just start using an SQLite database as a data format.
- jonstewart 4y agoSQLite is a row-oriented store, not a column-oriented store, ie, there are fundamental tradeoffs between the two.
- cm2187 4y agoBut sqlite has no compression by default.