zX41ZdbW

38 ms·

571.
by zX41ZdbW 4y ago
I briefly checked the dataset and I could not find the rank column. For example, the top 1000 URLs .txt file contains only the URLs itself, and it's not possible to check which is more popular: https://www.google.com or ht
572.
by zX41ZdbW 4y ago
Most of the time is spent in decompression - the source dataset used to have files in .bz2, which is the main contributor to total time. The dataset itself is just around 10 billion records.
573.
by zX41ZdbW 4y ago
Two SSDs on AWS machine only give 3800 MB/sec :(
574.
by zX41ZdbW 4y ago
In ClickHouse, querying Google Sheets works out of the box, no plugins required: select * from url('https://docs.google.com/spreadsheets/d/1XGCy0tYU5YcEouO09_ErZIyqjA-VJ4pidLZmMmJkEdk/gviz/tq?tq
575.
by zX41ZdbW 4y ago
> ClickHouse, for example, claims insert throughput performance of 50-200 MB/s and up to 1M+ rows per second in common use cases This is what I've written in my docs in 2016 :) Today you can easily get 50 million inserted recor
576.
by zX41ZdbW 4y ago
Is HN written in (Common (Lisp? Then it can start using (ClickHouse)))
577.
by zX41ZdbW 4y ago
We provide .deb, .rpm, .tgz, Docker, or single-binary, for x86-64, AArch64, for Linux, Mac and FreeBSD.
578.
by zX41ZdbW 4y ago
https://clickhouse.com/docs/en/install/#available-installati...
579.
by zX41ZdbW 4y ago
How does it look from command-line for streaming processing of CSV/TSV?
580.
by zX41ZdbW 4y ago
I recommend using clickhouse-local[1] for these tasks. It does SQL; it supports all imaginable data formats, streaming processing, and connecting to external data sources. It also outperforms every other tool[2]. [1] https://clic
581.
by zX41ZdbW 4y ago
The order should be as follows: - automated testing; - -Wall -Wextra, and -Weverything with some exceptions; - address, memory, thread, and undefined sanitizers; - fuzzing (at least some sort of), coverage-guided fuzzing with sanitizers; -
582.
by zX41ZdbW 4y ago
I'm using the same algorithm to color logs in ClickHouse: https://github.com/ClickHouse/ClickHouse/blob/82a6d75050513b... It also ignores Gamma, but at least it is described in the comments in the code.
583.
by zX41ZdbW 4y ago
This will save your time: https://play.clickhouse.com/play?user=play#U0VMRUNUIHRvcExld... Takeaways: - .org is 50% better than .com; - .edu and .gov are really nice; - .io is cool, and much better than .uk; PS. I don't
584.
by zX41ZdbW 4y ago
Just in case, Tinybird is built on top of ClickHouse. If you look deep enough into any analytics service, you'll find ClickHouse inside.
585.
by zX41ZdbW 4y ago
If you need to anonymize a dataset (structured, possibly linked tables), I recommend clickhouse-obfuscator - a tool designed specifically for this purpose: https://clickhouse.com/blog/five-methods-of-database-obfusca...
586.
by zX41ZdbW 4y ago
At least 1% of tweets are archived on Internet Archive. https://github.com/ClickHouse/ClickHouse/issues/46145
587.
by zX41ZdbW 4y ago
Interesting to compare the SWAR approach with 64-bit register, a similar approach with 128-bit register (SSE-2), and string search instructions (SSE-4.2). This implementation in ClickHouse can be used as a reference: https://gith
588.
by zX41ZdbW 4y ago
Let's check it with the OpenCellID dataset in ClickHouse: https://clickhouse.com/docs/en/getting-started/example-datas... Example: https://twitter.com/ClickHouseDB/status/160896
589.
by zX41ZdbW 4y ago
It sounds like CommonCrawl: https://commoncrawl.org/the-data/get-started/ You can download it, put it into ClickHouse, and get your own professional search engine. I've made up the term "professional sea
590.
by zX41ZdbW 4y ago
https://sql.clickhouse.com/play?user=play#U0VMRUNUIHRvU3Rhcn...
591.
by zX41ZdbW 4y ago
The number of comments per month continues to grow linearly, while the number of stories remains roughly constant for several years: https://sql.clickhouse.com/play?user=play#U0VMRUNUIHRvU3Rhcn...
592.
by zX41ZdbW 4y ago
What is the difference between Apache Doris, SelectDB and Starrocks? All of them pick the code from ClickHouse and take roots in Baidu PALO.
593.
by zX41ZdbW 4y ago
I've recently presented a talk about making a real-time dashboard on top of Wikipedia pageviews (0.5 trillion records) with a single machine, <2 bytes per record, and zero amount of sophistication: https://www.youtube.com
594.
by zX41ZdbW 4y ago
Sorry, but it does not work for 1.5 years already. Every time I check it does not work.
595.
by zX41ZdbW 4y ago
You don't have to give up clean code to achieve high performance. The complexity can be isolated and contained. For example, take a look at ClickHouse codebase: https://github.com/ClickHouse/ClickHouse/ There
596.
by zX41ZdbW 4y ago
When I tested BigQuery for ClickBench, the minimum query latency was 0.5 seconds. However, there is some feature for better query latency (need to check). Note: BigQuery has a "De-Witt" clause[2] that prevents publishing the bench
597.
ClickHouse to Serve Real-Time Queries on Top of BigQuery Data(clickhouse.com)
25 points by zX41ZdbW 4y ago|3 comments
598.
by zX41ZdbW 4y ago
The demo does not work at all: https://github.com/questdb/questdb/issues/1525
599.
by zX41ZdbW 4y ago
Case 1. It blocks my SQL playground due to SQL injections: https://play.clickhouse.com/play?user=play - solution: disable WAF. Case 2: It damages my presentations by removing whitespaces in HTML elements styled as "whi
600.
by zX41ZdbW 4y ago
The website does not open at this moment in time: https://downforeveryoneorjustme.com/exple.tive.org?proto=htt... About the topic: I always install pv, jq, dstat, perf, mc, rg, and clickhouse.

More ›