zX41ZdbW

40 ms·

541.
by zX41ZdbW 4y ago
I've added a benchmark of ClickHouse and Athena to ClickBench: https://pastila.nl/?0198061e/f2e0e7b2d61d0fe322607b58fc7200b... Where ClickHouse operates in a "data lake" mode - simply by processing a bun
542.
by zX41ZdbW 4y ago
Looks good, but more work is needed. The case of a suboptimal memory allocator - is the problem of shared linking. Ideally, we should make a "self-contained" shared library. It means that every dependent library, including malloc,
543.
by zX41ZdbW 4y ago
I recommend reading about a tool I've made specifically for this purpose: clickhouse-obfuscator: https://github.com/ClickHouse/ClickHouse/tree/master/program...
544.
by zX41ZdbW 4y ago
How large is the data size (the number of vectors, and their dimensions?), what are the type of queries (N nearest neighbors to a target vector according to L2 distance, or something else?), from where the queries are sent (reccomendation s
545.
by zX41ZdbW 4y ago
Also take a look at the recent project: NoiSQL — Generating Music With SQL Queries https://github.com/ClickHouse/NoiSQL
546.
by zX41ZdbW 4y ago
I've created my own list of lists as a pun for that: https://github.com/ClickHouse/ClickHouse/issues/29457
547.
by zX41ZdbW 4y ago
We have implemented asynchronous signal-safe in-process stack unwinding for always-on profiler in ClickHouse: https://clickhouse.com/docs/en/operations/optimizing-perform... The downside - it required many pa
548.
by zX41ZdbW 4y ago
If you write it as an SQL query in ClickHouse, it will take only 0.170 seconds: SELECT sum(reinterpretAsFloat32(number) BETWEEN 0.0 AND 1.0) FROM numbers_mt(0x100000000) 1 row in set. Elapsed: 0.170 sec. Processed 2.39 billion ro
549.
by zX41ZdbW 4y ago
It's too often the software from Ubuntu or Debian repositories is outdated. For example, Python, Docker, Rust, ClickHouse, Go, and Node.js - all have some versions in Debian, but the common knowledge is to ignore them and always instal
550.
by zX41ZdbW 4y ago
ClickHouse supports this and more: https://www.youtube.com/watch?v=zhrOYQpgvkk
551.
by zX41ZdbW 4y ago
ClickHouse has support for unsigned integer types: UInt8, UInt16, UInt32, UInt64, UInt128, and UInt256. The main applications are: hashes and identifiers, e.g., visitor identifier in clickstream.
552.
by zX41ZdbW 4y ago
Almost every web analytics, mobile analytics, and ad tech company uses ClickHouse. Examples: https://clickhouse.com/docs/en/about-us/adopters Datasets and blueprints: https://clickhouse.com/do
553.
by zX41ZdbW 4y ago
You can check how ClickHouse is cross-compiled for FreeBSD. 1. https://github.com/ClickHouse/sysroot/ 2. https://github.com/ClickHouse/ClickHouse/blob/master/cmake/f... an
554.
by zX41ZdbW 4y ago
It was an attempt to prepare a topic for a conference named "Doing weird stuff with ClickHouse" half a year ago. I already had an example of generating pictures, and some memory about bytebeat, so trying to generate sound was a na
555.
by zX41ZdbW 4y ago
Thank you! This topic is also new to me. When I was doing it, I had an impression that everyone else already knew about signal processing, electrical engineering, functional calculus, music theory, and sound engineering... While I only have
556.
Show HN: Building musical synthesizers with SQL queries(github.com)
176 points by zX41ZdbW 4y ago|14 comments
557.
by zX41ZdbW 4y ago
We use non-parametric statistics for performance testing in ClickHouse[1], picked from the article "A Randomized Design Used in the Comparison of Standard and Modified Fertilizer Mixtures for Tomato Plants". [1] https:/&#x2F
558.
by zX41ZdbW 4y ago
I've added two more examples yesterday (music4.sql and music5.sql): https://github.com/ClickHouse/NoiSQL/blob/main/music4.sql#L1... ChatGPT helps to overcome the shame of composing melodies by mysel
559.
Show HN: NoiSQL – Generating Music with SQL Queries(github.com)
4 points by zX41ZdbW 4y ago|1 comments
560.
by zX41ZdbW 4y ago
Interesting... I also thought about adding support for MYD format to ClickHouse. Because 15 years ago, the fastest way to insert data into MySQL was - to write a MYD file manually from C++ code into the MySQL data directory and then do some
561.
by zX41ZdbW 4y ago
Yes, the current experiment was run from a single VPC. It should explain the difference.
562.
by zX41ZdbW 4y ago
Worth to note that ClickHouse is a distributed MPP DBMS, it can scale up to 1000s of bare-metal servers and process terabytes (not terabits) of data per second. It also works on a laptop, or even without installation.
563.
by zX41ZdbW 4y ago
100 instances of m5.8xlarge, 500 instances of m5.2xlarge
564.
by zX41ZdbW 4y ago
No, I'm creating 10, 100, and 500 different EC2 instances and measuring the aggregate throughput.
565.
by zX41ZdbW 4y ago
https://repost.aws/knowledge-center/s3-maximum-transfer-spee...
566.
by zX41ZdbW 4y ago
https://imgur.com/a/uh8H9PO
567.
by zX41ZdbW 4y ago
We already partition our data correctly, as the AWS solution architects recommend, but there are limitations on the: - total throughput (100 Gbit/sec); - the number of requests. For example, at this moment, I'm doing an experiment
568.
by zX41ZdbW 4y ago
Ok, then here is the summary: We picked these ten technologies to translate YAML configs to cloud bills and put ClickHouse inside.
569.
by zX41ZdbW 4y ago
Yes, and actually, it is simpler than expected. The cached segments are written into the local filesystem, and therefore we get both the cache on SSD and the page cache in memory "for free". As ClickHouse is OLAP, the read/wr
570.
by zX41ZdbW 4y ago
I didn't know about bracketed paste until a few years ago. When I needed something similar in clickhouse-client, I implemented it as a heuristic - if the text is entered too quickly, it should be treated as pasted and there is no need

More ›