zX41ZdbW

55 ms·

751.
by zX41ZdbW 4y ago
https://github.com/ClickHouse/countwords/pull/1/files
752.
by zX41ZdbW 4y ago
I forgot to multiply the file 10 times. When I do, the result is 0.209 sec. which is still better than every other result.
753.
by zX41ZdbW 4y ago
I've checked with ClickHouse and the result is better than I expect... it runs in 0.043 sec. on my machine, which is faster than any other result. The code: SELECT arrayJoin(splitByChar(' ', lower(line))) AS word, count() AS
754.
by zX41ZdbW 4y ago
It's easy to download this data by yourself from HN API. Simple shell script and one day, and you have the full dataset: https://github.com/ClickHouse/ClickHouse/issues/29693
755.
by zX41ZdbW 4y ago
This looks surprisingly similar to the unfinished research that I did: https://github.com/ClickHouse/ClickHouse/issues/18842
756.
by zX41ZdbW 4y ago
DuckDB is included in this benchmark. But there were many OOMs in this benchmark and it was not easy to make it working: https://github.com/duckdb/duckdb/issues/3969 The data size is 75 GiB in uncompressed CS
757.
by zX41ZdbW 4y ago
I've also rerun the benchmark on 4x ra3.4xlarge (48 cores in total) and added it to the comparison.
758.
by zX41ZdbW 4y ago
ClickHouse can run in a classic shared-nothing setup and in "cloud-native" setup with shared storage. Setting up a distributed system with hundreds of machines can be difficult... but it's actually not more difficult than for
759.
by zX41ZdbW 4y ago
Databricks is in the list of systems I want to add: https://github.com/ClickHouse/ClickBench/#systems-included
760.
by zX41ZdbW 4y ago
ClickHouse is typically run on a cluster. There are setups with over 1000 machines and over 100 PB of data. But for this benchmark, the most simple setup is selected. I can add clustered setup for ClickHouse in this benchmark.
761.
by zX41ZdbW 4y ago
Redshift configuration was selected to get 16 cores in total, so it is 4 nodes of ra3.xplus 4vCPU to get 16vCPU in total. https://github.com/ClickHouse/ClickBench/tree/main/redshift We can add more resul
762.
by zX41ZdbW 4y ago
You are absolutely right. That's why this benchmark is named "a Benchmark For Analytical DBMS", not "the definitive benchmark for analytical DBMS".
763.
by zX41ZdbW 4y ago
There are many limitations of this benchmark, indeed: https://github.com/ClickHouse/ClickBench/#limitations
764.
by zX41ZdbW 4y ago
I've added BigQuery and run the benchmark and get the results... but due to their ToS , the benchmark results cannot be published. See https://cube.dev/blog/dewitt-clause-or-can-you-benchmark-a-d... The benchmark
765.
by zX41ZdbW 4y ago
I see that other systems often forgot many small details. Imagine you are implementing COUNT(DISTINCT) function. And imagine it is used in GROUP BY something, like GROUP BY region_id. And on realistic datasets, you will have a large number
766.
by zX41ZdbW 4y ago
Yes, this is what I want to include - the cost metric. Although it will be tricky for systems with consumption-based pricing: BigQuery and Athena. And Snowflake has almost consumption based pricing - while you pay for warehouse time, it can
767.
Show HN: A benchmark for analytical databases (Snowflake, Druid, Redshift)(benchmark.clickhouse.com)
137 points by zX41ZdbW 4y ago|57 comments
768.
by zX41ZdbW 4y ago
I recommend checking out clickhouse-obfuscator. It's a more sophisticated tool for dataset obfuscation. Installation (single binary Linux/Mac/FreeBSD): curl https://clickhouse.com/ | sh ./clickhouse obfu
769.
by zX41ZdbW 4y ago
ClickHouse has SIMD implementation for lower/upper for ASCII and UTF-8. It can easily make ~20 GB/sec with 48 cores: https://pastila.nl/?0f15f812/179361234a6b79d27e18c8db7050bd0... Although it is only using
770.
by zX41ZdbW 4y ago
I want to try DuckDB but it fails to load a Parquet file: https://github.com/duckdb/duckdb/issues/3969
771.
by zX41ZdbW 4y ago
If you need customizable reports (filtering and aggregating by any dimension), you cannot avoid scan over non-aggregated data. This is where you cannot optimize Postgres queries. Postgres becomes slow starting from just single million recor
772.
by zX41ZdbW 4y ago
It is shipped with debug info and symbol tables just in case. Without debug info, it will be 350 MB, and compressed can fit in 50 MB: https://github.com/ClickHouse/ClickHouse/issues/29378 It is definitely a w
773.
ClickHouse over the Years with Benchmarks(clickhouse.com)
1 points by zX41ZdbW 4y ago|0 comments
774.
by zX41ZdbW 4y ago
Maybe Cosmopolitan Libc can be developed to become a full-featured glibc replacement like Musl and what LLVM libc is going to target? To be able to link with large C++ codebases it is still missing a ton of small cruft...
775.
by zX41ZdbW 4y ago
It does not look really convincing if you look deeper: https://telegra.ph/No-QuestDB-is-not-Faster-than-ClickHouse-...
776.
by zX41ZdbW 4y ago
About the system.asynchronous_metric_log table. It's very simple. Every ClickHouse instance, even empty, is collecting system-wide monitoring information. You can query this table directly from Grafana dashboard (for example).
777.
by zX41ZdbW 4y ago
I recommend to write a follow-up with the topics: 1. clickhouse-git-import tool, what is it and how to use it. 2. ClickHouse as a monitoring agent with the system.asynchronous_log table. 3. Hosting static MergeTree tables on any web server.
778.
by zX41ZdbW 4y ago
In clickhouse-local example, you can omit the --structure argument and it will continue to work with automatic schema inference.
779.
by zX41ZdbW 4y ago
I have a project that allows not only to compare the stars, but also to find similar repositories, trends and interesting facts: https://ghe.clickhouse.tech/
780.
by zX41ZdbW 4y ago
The calculation method and the overall page design is very similar to ClickHouse benchmarks: https://clickhouse.com/benchmark/dbms/ https://presentations.clickhouse.com/original_website/benchm

More ›