2 ms·
So in this example the image says "lineitem_1t_parq" which makes me think that's probably parquet files (optimized and perhaps compressed columnar file format).
by makmanalp 10y ago
So in this example the image says "lineitem_1t_parq" which makes me think that's probably parquet files (optimized and perhaps compressed columnar file format).
But it does the same stuff with CSVs too, perhaps a bit less efficiently. JSON is also definitely possible, but a bit more tricky because of the inherently nested structure and more complex format.
Take a look at this for the details on a similar idea: http://stratos.seas.harvard.edu/files/stratos/files/nodb-cacm.pdf http://stratos.seas.harvard.edu/files/stratos/files/nodb-cac...
- kakoni 10y agoThat paper discusses PostgresRAW, which can be found here; https://github.com/HBPMedical/PostgresRAW https://github.com/HBPMedical/PostgresRAW I wonder if there are any similar projects using psql as an engine?
- makmanalp 10y agoI think it's pretty common - it's open source, nice and modular codebase, reasonably advanced execution engine with stuff like MVCC. But also redshift itself, citus, hadapt and greenplum are examples that come to my mind.