DuckDB also compresses the data. I've ingested ≈ 1 TB of apache-like access logs into a duckdb file that was small enough to load completely into ram on my loca

DuckDB also compresses the data. I've ingested ≈ 1 TB of apache-like access logs into a duckdb file that was small enough to load completely into ram on my local machine (like 10-15 GB). It was blazingly fast to query over. (The large spark cluster was still ingesting the data when I was done with my analysis, only working on my local laptop)