5 ms·
I am also interested in writing the SQLite or PostgreSQL file format straight to disk as a faster way to do ETL. I'd be curious to hear if anyone has actually t
by eatonphil 5y ago
I am also interested in writing the SQLite or PostgreSQL file format straight to disk as a faster way to do ETL. I'd be curious to hear if anyone has actually tried this. I'd be as curious if you end up trying this too.
- tyingq 5y agoSqlite has a btree module/api, though there's a lot of "TODO:" notes in the document: https://sqlite.org/btreemodule.html https://sqlite.org/btreemodule.html
- avinassh 5y agoTIL, this is great! I think this could make things even easier instead of writing the raw pages
- oldtablet 5y agoI really like sqlite but I'm curious how fast it would be to write the data to parquet files and read it with duckdb. I've done something similar before and in my case duckdb was faster than sqlite.
- avinassh 5y agoyes, I plan to try this next and measure the performance. Others also suggested similar idea to try. > I am also interested in writing the SQLite or PostgreSQL file format straight to disk as a faster way to do ETL. what exactly you are trying to do here?
- deleted 5y ago[deleted]
- exikyut 5y agoSee also https://news.ycombinator.com/item?id=27880920 https://news.ycombinator.com/item?id=27880920