6 ms·

From my practical experience with ClickHouse, I saw OOMs as well, on load and on query. How confident are you that your chosen dataset is neutral?

by vgt 4y ago

zX41ZdbW 4y ago
ClickHouse will throw an exception in case of not enough memory and continue to serve other queries. Under certain configurations it can OOM as well. > How confident are you that your chosen dataset is neutral? I have no idea if it is "neutral", I picked it randomly. I test ClickHouse on every interesting dataset, see here: https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aissue+label%3Adataset https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aissue... The reason - I love working with data :) If I see a dataset, I load it into ClickHouse - this is the first thing I do. This is not a kind of marketing or promotion of ClickHouse - you know, if it were some directed task, it would be uninteresting for me.
- vgt 4y ago
  wonderful to hear! So any word on correctness or nah?
  - zX41ZdbW 4y ago
    I did say about > Query language, data types support, feature completeness, stability and testing nothing about correctness. In terms of stability, I see a couple of pretty old, and still unresolved issues about memory safety (data races, segmentation faults) in your repository, found by users. In contrast, most of the memory safety issues in ClickHouse are found by continuous fuzzing before the release. And finding similar issues will give you a reward: https://github.com/ClickHouse/ClickHouse/issues/38986 https://github.com/ClickHouse/ClickHouse/issues/38986 Our testing system successfully finding issues in well known and widely used libraries - jemalloc, rocksdb, grpc, AWS, Arrow, Avro, ZooKeeper, Linux kernel... It is kind of surprising, and it makes an impression like we are the only product that does testing for real. I also remember an example of using SQLancer from 1.5 years ago. When SQLancer appeared, we started to use it on ClickHouse, and it has found a few issues and one crash. At the same time, it has found a lot of crashes in DuckDB. But this example is very old, and DuckDB evolved a lot since then - it is a much younger technology after all.

From my practical experience with ClickHouse, I saw OOMs as well, on load and on query. How confident are you that your chosen dataset is neutral?

by vgt 4y ago

From my practical experience with ClickHouse, I saw OOMs as well, on load and on query. How confident are you that your chosen dataset is neutral?

zX41ZdbW 4y ago

ClickHouse will throw an exception in case of not enough memory and continue to serve other queries. Under certain configurations it can OOM as well. > How confident are you that your chosen dataset is neutral? I have no idea if it is "neutral", I picked it randomly. I test ClickHouse on every interesting dataset, see here: https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aissue+label%3Adataset https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aissue... The reason - I love working with data :) If I see a dataset, I load it into ClickHouse - this is the first thing I do. This is not a kind of marketing or promotion of ClickHouse - you know, if it were some directed task, it would be uninteresting for me.

vgt 4y ago
wonderful to hear! So any word on correctness or nah?
- zX41ZdbW 4y ago
  I did say about > Query language, data types support, feature completeness, stability and testing nothing about correctness. In terms of stability, I see a couple of pretty old, and still unresolved issues about memory safety (data races, segmentation faults) in your repository, found by users. In contrast, most of the memory safety issues in ClickHouse are found by continuous fuzzing before the release. And finding similar issues will give you a reward: https://github.com/ClickHouse/ClickHouse/issues/38986 https://github.com/ClickHouse/ClickHouse/issues/38986 Our testing system successfully finding issues in well known and widely used libraries - jemalloc, rocksdb, grpc, AWS, Arrow, Avro, ZooKeeper, Linux kernel... It is kind of surprising, and it makes an impression like we are the only product that does testing for real. I also remember an example of using SQLancer from 1.5 years ago. When SQLancer appeared, we started to use it on ClickHouse, and it has found a few issues and one crash. At the same time, it has found a lot of crashes in DuckDB. But this example is very old, and DuckDB evolved a lot since then - it is a much younger technology after all.