4 ms·

How does it compare to duckdb and/or polars?

thenaturalist 1y ago
This is very much an active space, so the half-life of in depth analyses is limited, but one of the best write ups from about 1.5 years ago is this one: https://bicortex.com/duckdb-vs-clickhouse-performance-comparison-for-structured-data-serialization-and-in-memory-tpc-ds-queries-execution/ https://bicortex.com/duckdb-vs-clickhouse-performance-compar...
nasretdinov 1y ago
In my understanding DuckDB doesn't have its own optimised storage that can accept writes (in a sense that ClickHouse does, where it's native storage format gives you best performance), and instead relies on e.g. reading data from Parquet and other formats. That makes sense for an embedded analytics engine on top of existing files, but might be a problem if you wanted to use DuckDB e.g. for real-time analytics where the inserted data needs to be available for querying in a few seconds after it's been inserted. ClickHouse was designed for the latter use case, but at a cost of being a full-fledged standalone service by design. There are embedded versions of ClickHouse, but they are much bulkier and generally less ergonomic to use (although that's a personal preference)
- theLiminator 1y ago
  Nah, duckdb does have their own format (not sure if it's write friendly, though i believe it is).
  - nasretdinov 1y ago
    It does, but the performance isn't great apparently: https://github.com/duckdb/duckdb/discussions/10161 https://github.com/duckdb/duckdb/discussions/10161
    - theLiminator 1y ago
      Yeah, I don't really know. Though in the OLAP space that issue/discussion is really old. There's a good chance that performance is dramatically better now though YMMV.
  - thenaturalist 1y ago
    It cannot do concurrent writes natively and that is not a design goal of the DuckDB creators. See: https://duckdb.org/docs/stable/connect/concurrency.html#writing-to-duckdb-from-multiple-processes https://duckdb.org/docs/stable/connect/concurrency.html#writ...
spatular 1y ago
Clickhouse is a network server, duckdb and polars are in-process databases. It's like postgres vs sqllite.
- theLiminator 1y ago
  There's chdb though.

This is very much an active space, so the half-life of in depth analyses is limited, but one of the best write ups from about 1.5 years ago is this one: https://bicortex.com/duckdb-vs-clickhouse-performance-comparison-for-structured-data-serialization-and-in-memory-tpc-ds-queries-execution/ https://bicortex.com/duckdb-vs-clickhouse-performance-compar...

nasretdinov 1y ago

In my understanding DuckDB doesn't have its own optimised storage that can accept writes (in a sense that ClickHouse does, where it's native storage format gives you best performance), and instead relies on e.g. reading data from Parquet and other formats. That makes sense for an embedded analytics engine on top of existing files, but might be a problem if you wanted to use DuckDB e.g. for real-time analytics where the inserted data needs to be available for querying in a few seconds after it's been inserted. ClickHouse was designed for the latter use case, but at a cost of being a full-fledged standalone service by design. There are embedded versions of ClickHouse, but they are much bulkier and generally less ergonomic to use (although that's a personal preference)

theLiminator 1y ago
Nah, duckdb does have their own format (not sure if it's write friendly, though i believe it is).
- nasretdinov 1y ago
  It does, but the performance isn't great apparently: https://github.com/duckdb/duckdb/discussions/10161 https://github.com/duckdb/duckdb/discussions/10161
  - theLiminator 1y ago
    Yeah, I don't really know. Though in the OLAP space that issue/discussion is really old. There's a good chance that performance is dramatically better now though YMMV.
- thenaturalist 1y ago
  It cannot do concurrent writes natively and that is not a design goal of the DuckDB creators. See: https://duckdb.org/docs/stable/connect/concurrency.html#writing-to-duckdb-from-multiple-processes https://duckdb.org/docs/stable/connect/concurrency.html#writ...

spatular 1y ago

Clickhouse is a network server, duckdb and polars are in-process databases. It's like postgres vs sqllite.

theLiminator 1y ago
There's chdb though.