3 ms·

The best feature of Clickhouse is replacing entire data engineering and analytics teams and their “modern data platform” with one person and one application.

deleted 4y ago
[deleted]
polskibus 4y ago
How’s that? How does clickhouse replace ETLs? How does it alleviate data quality problems? It is just a data warehouse , very fast but with limited data modelling and permissioning compared to MS OLAP or tabular models.
- deleted 4y ago
  [deleted]
- benjaminwootton 4y ago
  Im not really disagreeing with him, but you can use DBT to materialise 10 daisy chained views, or hit Clickhouse with a query and allow it to crunch data. I am a proponent of modern data stack but it’s always at the back of my mind that an OLAP database on a big box negates the need for a lot of it.
  - polskibus 4y ago
    Sure it does, but you have to feed it somehow and maintain it so that the data there is of good quality and makes business sense. Afaik, clickhouse is not great for multitable joins required by good data warehousing practices like star and snowflake schema.
    - gauravphoenix 4y ago
      CH is actually very good if you can model the dimensional data as dictionary. Also CH can use external store like PSQL as dictionary. But yes, if your fact table data size is as big as dimensional data, CH or any other such system will be quite slow.
- datalopers 4y ago
  ETL: Clickhouse can probably directly read any format or data source you have. Quality: A diligent human like anywhere. Data Modeling: I disagree, what do you think it is missing? I’ve never encountered anything I couldn’t achieve, and the Clickhouse specific aggregates, lambda support, and array functionality really simplifies many problems. Permissioning: It has very granular permissions.
- gauravphoenix 4y ago
  If you use the "Object" data type (json), you can have a schema-less datastore and that can alleviate the need of complex ETL pipelines. But yes, if you need to decorate the data before ingestion, you need some ETL pipeline. In other words, if the ETL pipeline is just storing the data in columnar format by reading some JSON fields, you don't need that anymore with the new CH json data type

deleted 4y ago

[deleted]

polskibus 4y ago

How’s that? How does clickhouse replace ETLs? How does it alleviate data quality problems? It is just a data warehouse , very fast but with limited data modelling and permissioning compared to MS OLAP or tabular models.

deleted 4y ago
[deleted]
benjaminwootton 4y ago
Im not really disagreeing with him, but you can use DBT to materialise 10 daisy chained views, or hit Clickhouse with a query and allow it to crunch data. I am a proponent of modern data stack but it’s always at the back of my mind that an OLAP database on a big box negates the need for a lot of it.
- polskibus 4y ago
  Sure it does, but you have to feed it somehow and maintain it so that the data there is of good quality and makes business sense. Afaik, clickhouse is not great for multitable joins required by good data warehousing practices like star and snowflake schema.
  - gauravphoenix 4y ago
    CH is actually very good if you can model the dimensional data as dictionary. Also CH can use external store like PSQL as dictionary. But yes, if your fact table data size is as big as dimensional data, CH or any other such system will be quite slow.
datalopers 4y ago
ETL: Clickhouse can probably directly read any format or data source you have. Quality: A diligent human like anywhere. Data Modeling: I disagree, what do you think it is missing? I’ve never encountered anything I couldn’t achieve, and the Clickhouse specific aggregates, lambda support, and array functionality really simplifies many problems. Permissioning: It has very granular permissions.
gauravphoenix 4y ago
If you use the "Object" data type (json), you can have a schema-less datastore and that can alleviate the need of complex ETL pipelines. But yes, if you need to decorate the data before ingestion, you need some ETL pipeline. In other words, if the ETL pipeline is just storing the data in columnar format by reading some JSON fields, you don't need that anymore with the new CH json data type