4 ms·

What do you mean? Does it have an etl engine like MS SSIS or scheduler like airflow built-in?

• nojito 5y ago
  I essentially load in two columns one called timestamp and another with a json blob. I then use this to kick off materialized views to automagically pluck out relevant JSON fields into views Similar to this https://eng.uber.com/logging/ https://eng.uber.com/logging/
  • flurly 5y ago
    Great minds think alike :) We do the exact same thing at GraphJSON https://www.graphjson.com/guides/about https://www.graphjson.com/guides/about
  • mr-karan 5y ago
    That's an interesting way to load data and then use with Materialized Views. However, I am curious how do you make efficient use of compression codecs[0] that Clickhouse provides, or some neat features like TTL policies [1] using this method? [0]: https://clickhouse.com/docs/en/sql-reference/data-types/lowcardinality/ https://clickhouse.com/docs/en/sql-reference/data-types/lowc... [1]: https://clickhouse.com/docs/en/sql-reference/statements/alter/ttl/ https://clickhouse.com/docs/en/sql-reference/statements/alte...
    • Redsquare 5y ago
      Materialized views have a backing table where you can use the codecs. Can add ttl the ingestion table
      • mr-karan 5y ago
        The backing table in OP's case is just 2 columns, timestamp and a `String` column representing JSON blob. Can't do much there.
        • Redsquare 5y ago
          He said the injest table is defined as that not the MV backing table