4 ms·

thanks for the note. makes a lot of sense. >I want a more OLAP style querying than OLTP and I know my data then today ClickHouse absolutely shines here with th

thanks for the note. makes a lot of sense. >I want a more OLAP style querying than OLTP and I know my data then today ClickHouse absolutely shines here with the others playing catch up Curious, what type of OLAP workloads did you work with? Were there cases where you tried both CH and Elastic for the same use case and decided for oner over the other?

buro9 4y ago
Again - "it depends". The decisions I've seen firsthand (or been a part of): CH vs ES: In petabyte sized stores of highly homogenised data (very few well structured types of data, i.e. "HTTP requests" where all logs were precisely following the same format and blended business data (i.e. the HTTP logs contained info on customer, tenant, etc)) and the primary goal was business insight and operator analytics - ClickHouse was the clear winner (capability to ingest at this scale whilst providing query performance at this scale). This was Cloudflare for the HTTP logs, L7 firewall logs and the L3 logs (sampled packet headers). They've blogged about them enough that there's no secrecy they are using CH for this. Loki vs CH: In petabyte sized stores of logs from tens to hundreds of thousands of machines where every log is differently structured across apps, OS, device where the primary goal was to store all logs and provide operator observability - Loki was the clear winner (capability to ingest, control over storage costs). There is no one-size-fits-all, and benchmarks are not useful in answering the questions that should be asked. Engineers should go back to first principles and evaluate systems and solutions according to their specific criteria. Some may even end on an entirely different solution than the ones you've outlined - PostgreSQL is pretty phenomenal too, and the full text search on that is great.

4 ms·

thanks for the note. makes a lot of sense. >I want a more OLAP style querying than OLTP and I know my data then today ClickHouse absolutely shines here with th

by pranay01 4y ago

buro9 4y ago
Again - "it depends". The decisions I've seen firsthand (or been a part of): CH vs ES: In petabyte sized stores of highly homogenised data (very few well structured types of data, i.e. "HTTP requests" where all logs were precisely following the same format and blended business data (i.e. the HTTP logs contained info on customer, tenant, etc)) and the primary goal was business insight and operator analytics - ClickHouse was the clear winner (capability to ingest at this scale whilst providing query performance at this scale). This was Cloudflare for the HTTP logs, L7 firewall logs and the L3 logs (sampled packet headers). They've blogged about them enough that there's no secrecy they are using CH for this. Loki vs CH: In petabyte sized stores of logs from tens to hundreds of thousands of machines where every log is differently structured across apps, OS, device where the primary goal was to store all logs and provide operator observability - Loki was the clear winner (capability to ingest, control over storage costs). There is no one-size-fits-all, and benchmarks are not useful in answering the questions that should be asked. Engineers should go back to first principles and evaluate systems and solutions according to their specific criteria. Some may even end on an entirely different solution than the ones you've outlined - PostgreSQL is pretty phenomenal too, and the full text search on that is great.