3 ms·
Perhaps slightly off topic, but I'm not familiar with DuckDB. I don't really understand what it is for, and their site is doesn't really explain it well either.
by sspiff 3y ago
Perhaps slightly off topic, but I'm not familiar with DuckDB. I don't really understand what it is for, and their site is doesn't really explain it well either.
What is the advantage of using this instead of something well-proven and battle tested like an in memory SQLite database?
- victorbjorklund 3y agoDuckDB is a columnar db and focused on analysis queries. So it is more optimized for those workflows. Of course for many things sqlite is fine but if you have a really large db with complex queries duckdb should be faster.
- filleokus 3y agoDuckDB also compresses the data. I've ingested ≈ 1 TB of apache-like access logs into a duckdb file that was small enough to load completely into ram on my local machine (like 10-15 GB). It was blazingly fast to query over. (The large spark cluster was still ingesting the data when I was done with my analysis, only working on my local laptop)