zX41ZdbW

29 ms·

421.
by zX41ZdbW 3y ago
It is the case when a library already uses dlopen in unusual code paths, but we want to make sure that code paths won't work. The examples are some authentication plugins.
422.
by zX41ZdbW 3y ago
It is mostly to protect from unusual configurations rather than protecting from an attacker, e.g., https://github.com/ClickHouse/ClickHouse/issues/43933 and https://github.com/ClickHouse/
423.
by zX41ZdbW 3y ago
In ClickHouse, we forbid LD_PRELOAD and every similar variable: https://github.com/ClickHouse/ClickHouse/blob/master/program... We also forbid dlopen - so even if some third-party library commits such an
424.
by zX41ZdbW 3y ago
It's sad that duckdb timed out on R2 and crashed with OOM on a simple query, but not to expect much. I think ClickHouse should do well on this test. Five billion records are very few. How can I get the dataset to test?
425.
by zX41ZdbW 3y ago
It is the same method that I use in clickhouse-obfuscator: https://github.com/ClickHouse/ClickHouse/tree/master/program...
426.
by zX41ZdbW 3y ago
Or Cache Oblivious Lookahead Arrays which considered to be an alternative to MergeTree (LSM), but not used anymore: https://github.com/giannitedesco/cola
427.
by zX41ZdbW 3y ago
This reminded me about https://github.com/cdnjs/cdnjs/ - every version of every popular JS library in one repository - one of the largest repositories on GitHub by size.
428.
by zX41ZdbW 3y ago
Although the same query can be run directly from Parquet files with clickhouse-local: clickhouse-local --query "SELECT project_name, sum(lines), sum(size) FROM '*.parquet' WHERE skip_reason = '' AND project_na
429.
by zX41ZdbW 3y ago
I've uploaded the metadata to ClickHouse Playground for analysis. Example: https://play.clickhouse.com/play?user=play#U0VMRUNUIHByb2plY... This is a nice dataset for exploration, although small (only one billion record
430.
by zX41ZdbW 3y ago
For low-level sound generation, I'd also recommend checking out my experimental project https://github.com/ClickHouse/NoiSQL It's unusual because it uses only SQL.
431.
by zX41ZdbW 3y ago
The best article I've seen on this topic is https://www.handprint.com/HP/WCL/color1.html Warning: super-long read, you will spend a day. You'll be surprised, but I use some thoughts from this article in
432.
by zX41ZdbW 3y ago
One of the features I wish Postgres had is "column matchers" from ClickHouse: https://clickhouse.com/docs/en/sql-reference/statements/sele... SELECT COLUMNS('_id$') FROM table;
433.
by zX41ZdbW 3y ago
We are choosing between Darling and using Mac instances in AWS for validation of ClickHouse builds: https://github.com/ClickHouse/ClickHouse/issues/15609 The fact that Darling requires a kernel module makes i
434.
by zX41ZdbW 3y ago
ClickHouse has two modes of operation on S3: 1. S3 as main storage with a write-through cache. 2. S3 as a cold tier in tiered storage. It works well because the data is organized by a set of immutable parts called MergeTree. These data part
435.
by zX41ZdbW 3y ago
clickhouse-format does it better: SELECT COUNT(*) FROM table ; SELECT * FROM table WHERE (frozen_rand >= 123) AND (frozen_rand <= 456) ORDER BY RAND() ASC LIMIT 1000 ; It also has the "ob
436.
by zX41ZdbW 3y ago
Here is a good collection with ready-to-use instructions and downloads: https://clickhouse.com/docs/en/getting-started/example-datas...
437.
by zX41ZdbW 3y ago
There is no need for specialized time-series databases, because normal OLAP databases work better for time-series. I see it from my experience with ClickHouse. It might sound paradoxical, but specialized time-series databases, such as Times
438.
by zX41ZdbW 3y ago
ClickHouse uses "grace hash" GROUP BY with the number of buckets = 256. It can do size about 256 times larger than a memory because only one bucket has to be in memory while merging. It works for distributed query processing as we
439.
by zX41ZdbW 3y ago
ClickHouse can do large GROUP BY queries, not limited by memory: https://clickhouse.com/docs/en/sql-reference/statements/sele...
440.
by zX41ZdbW 3y ago
We initially had a pull-based query engine in ClickHouse, but then migrated to a dataflow graph query engine: https://github.com/ClickHouse/ClickHouse/blob/master/src/Pro... It allows decoupling of
441.
by zX41ZdbW 3y ago
Duplicate of https://news.ycombinator.com/item?id=36671322
442.
by zX41ZdbW 3y ago
Interesting that Hive and HDFS integration is available in ClickHouse for a long time. You can check the comparison with BigQuery here: https://clickhouse.com/blog/clickhouse-bigquery-migrating-da...
443.
by zX41ZdbW 3y ago
MsgPack is >1.5 times faster than JSONL: https://pastila.nl/?003d59f7/15a4f09bfb4c44ca92082bd5e54d8b7...
444.
by zX41ZdbW 3y ago
We support CapNProto as an import/export format in ClickHouse. It was contributed by Marek Vavruša from CloudFlare in 2017: https://github.com/ClickHouse/ClickHouse/pull/1387 Since then, we extended and
445.
by zX41ZdbW 3y ago
I did a performance comparison of Redshift and Athena (including Redshift classic and serverless) in ClickBench: https://benchmark.clickhouse.com/ It has an open methodology, and anyone can reproduce the results.
446.
by zX41ZdbW 3y ago
A nice writeup: https://notes.volution.ro/v1/2022/01/notes/fbf3f06c/
447.
by zX41ZdbW 3y ago
I'm amazed at how quickly the author responds to requests: https://github.com/rui314/mold/issues/1057 From the report to the fix in less than two days. I'm not sure how competitive it will be with l
448.
by zX41ZdbW 3y ago
We are doing cross-compilation, and all the targets can be built on x86_64 or aarch64 servers or laptops/desktops; in the CI, we are using x86_64 machines in AWS.
449.
by zX41ZdbW 3y ago
> Can you embed it in Python as a library? https://github.com/chdb-io/chdb pip install chdb
450.
by zX41ZdbW 3y ago
Yes: clickhouse-local --dialect prql or: $ clickhouse-local :) SET dialect = 'prql'

More ›