zX41ZdbW

39 ms·

1. 511.▲
   by zX41ZdbW 3y ago

   Good instruction on building your own sensor here: https://sensor.community/en/sensors/airrohr/ As a bonus, the collected data by the sensors.community project is public, and here is the walkthrough of data a
2. 512.▲
   by zX41ZdbW 3y ago

   Let's add it to ClickHouse: https://github.com/ClickHouse/ClickHouse/blob/master/base/ba... It should significantly improve the performance on ARM.
3. 513.▲
   by zX41ZdbW 3y ago

   Runtime CPU dispatching: https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html
4. 514.▲
   by zX41ZdbW 3y ago

   ClickHouse uses jemalloc as the only option: https://github.com/ClickHouse/ClickHouse/ We have contributed patches and bugfixes to jemalloc as well.
5. 515.▲
   by zX41ZdbW 3y ago

   ClickHouse cluster for web analytics data, 120 trillion records, mostly in two tables (hits and visits). 133 PiB uncompressed / 8.31 PiB compressed. But it is not large compared to what other ClickHouse users have, for example: > Ah
6. 516.▲
   by zX41ZdbW 3y ago

   This is amazing news! PS. Interesting if someone has experience integrating Go Asm dialect into C or C++ build system? We needed it for another experiment https://github.com/ClickHouse/ClickHouse/issues/45130#
7. 517.▲
   by zX41ZdbW 3y ago

   You can check my presentation on this topic. It covers the case when compression has to be faster than memory bandwidth: https://presentations.clickhouse.com/meetup53/optimizations/
8. 518.▲
   by zX41ZdbW 3y ago

   It looks similar to LZSSE. We tried it in ClickHouse, but then removed it: https://github.com/ClickHouse/ClickHouse/pull/24424 Reasons: - the decompression speed is slightly better than for lz4, but the compr
9. 519.▲
   by zX41ZdbW 3y ago

   It's not correct to say that relational DBMS are not suitable for time series. ClickHouse is a relational DBMS, and it works for time series better than specialized systems (like TimescaleDB or InfluxDB). You can quickly get trillions
10. 520.▲
    by zX41ZdbW 3y ago

    You can download the dataset of dislikes and analyze it in ClickHouse: https://clickhouse.com/docs/en/getting-started/example-datas...
11. 521.▲
    by zX41ZdbW 3y ago

    The easiest and best way is to use ClickHouse. It installs as a single binary, reads every data format, interacts with Postgres, MySQL, and even SQL Server, and has the most efficient and versatile SQL engine. At this moment in time, nothin
12. 522.▲
    by zX41ZdbW 3y ago

    ClickHouse allows to running Python code inside the database for ML. See the presentation for examples: https://presentations.clickhouse.com/meetup74/ai/
13. 523.▲
    by zX41ZdbW 3y ago

    Duplicate of https://news.ycombinator.com/item?id=35982077
14. 524.▲
    by zX41ZdbW 3y ago

    That's true - neither of the three data lake formats has good C++ or Rust libraries. We have to support all of them (Iceberg, Hudi, and Delta Lake) in ClickHouse, but afford it only for reading. The overall data lakes infrastructure is
15. 525.▲
    by zX41ZdbW 3y ago

    Duplicate of https://news.ycombinator.com/item?id=35982077
16. 526.▲
    by zX41ZdbW 3y ago

    You can load the data into ClickHouse for analysis: https://github.com/ClickHouse/ClickHouse/issues/22482#issuec...
17. 527.▲
    by zX41ZdbW 3y ago

    It can be applied for a larger dataset, see https://github.com/ClickHouse/ClickBench/tree/main/clickhous... (100 billion records), but so far only ClickHouse has been tested on this volume - it takes too
18. 528.▲
    by zX41ZdbW 3y ago

    I am the author of ClickBench: https://github.com/ClickHouse/ClickBench/ You can check the methodology described in the README and check the pull requests submitted by many database vendors.
19. 529.▲
    by zX41ZdbW 3y ago

    I recommend checking ClickBench: https://github.com/ClickHouse/ClickBench Disclaimer: I'm the author of ClickBench. It contains the comparison of Datafusion, DuckDB as well, and 30+ more DB engines in an open benc
20. 530.▲
    by zX41ZdbW 3y ago

    Duplicate of https://news.ycombinator.com/item?id=35982077
21. 531.▲
    by zX41ZdbW 3y ago

    Just leave everything by default - no need specifying any codecs, etc. People often try to show up their knowledge by using advanced features, but that's not good.
22. 532.▲
    by zX41ZdbW 3y ago

    I forgot to mention that ClickHouse works on ARM. We build two binaries for ARM: one for modern AArch64 servers and another for older and smaller devices without the support for the latest CPU instructions.
23. 533.▲
    by zX41ZdbW 3y ago

    I think ClickHouse should fit. There are many examples of its usage at the edge: https://clickhouse.com/docs/en/about-us/adopters E.g., FastNetMon: https://fastnetmon.com/docs-fnm-advanced&#x2
24. 534.▲
    by zX41ZdbW 3y ago

    Even the stripes in the new tiger-looking logo are the same as in ClickHouse. So bizarre - why are they pretending so much?
25. 535.▲
    by zX41ZdbW 3y ago

    Yes, here is the article about the techniques: https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html
26. 536.▲
    by zX41ZdbW 3y ago

    Here is a discussion about introducing AVX by default in ClickHouse: https://github.com/ClickHouse/ClickHouse/issues/40459 TLDR: runtime CPU dispatching and JIT - ok, AVX by default - not ok.
27. 537.▲
    by zX41ZdbW 3y ago

    I found many ClickHouse-related projects in a dedicated cluster - "House of Clicks" :) But ClickHouse itself somehow appears in Kubernation, and ClickBench - in Datapolis. Nice names btw.
28. 538.▲
    by zX41ZdbW 3y ago

    I recommend checking https://ghe.clickhouse.tech/ It explains the full pipeline - how to download, collect, and analyze this sort of data.
29. 539.▲
    by zX41ZdbW 3y ago

    It's easy to download this data by yourself from HN API. Simple shell script and one day, and you have the full dataset: https://github.com/ClickHouse/ClickHouse/issues/29693
30. 540.▲
    by zX41ZdbW 3y ago

    Tried Unity builds recently for ClickHouse, but without success: https://github.com/ClickHouse/ClickHouse/pull/18952#issuecom...