4 ms·
I'm relatively new to both and love them both for what they do well. Duck is "in process" like SQLite. Clickhouse is a client/server model like most other dat
by Jgrubb 2y ago
I'm relatively new to both and love them both for what they do well. Duck is "in process" like SQLite. Clickhouse is a client/server model like most other databases.
Duck is currently more optimized for the analyst or whomever working locally, importing and cleaning data - it's incredibly generous and well optimized for importing many different formats and handling all manner of CSV varieties etc. It's fast and fun to work with but being in process it's not optimized for, say, setting up remotely as the backend for a BI thing like Metabase and then running typical ETL processes to bring in new data. Only 1 process is allowed to connect at a time, so Metabase would sit on that and then you can't import new data in another process. (There is a RO connect mode where multiple processes can connect to read, but nothing is allowed to write if something else is connected.)
Clickhouse feels more like a traditional data warehouse solution, but wicked fast. It's a lot more fussy with getting data in - it's not nearly as generous with importing CSV for example - but once it's in and in the CH native format it is jaw-droppingly fast. Its compression scheme(s) is super efficient and it's also quite fun to play with.
I'm on a BigQuery backed data stack at my job but dream about usecases where we might employ Clickhouse for something more specialized. I use Duck when I'm building an analysis - rather than constantly connecting to BQ I'll just bring the base data into DuckDB locally and then work from there.
I also use them together on a side project of mine - I eat questionable CSV with Duck, then save it to Parquet which Clickhouse will easily handle.
- datadrivenangel 2y agoI'm really excited for MotherDuck, which is basically going to be a BigQuery style serverless data warehouse, but DuckDB flavored, so you can do the local development and then just move it to the cloud without having to translate.
- Vaslo 2y agoInteresting! I've been thinking about what you are doing but instead using DuckDB and Postgres but maybe I'll give Clickhouse a look instead of Postgres.