4 ms·
Yes, time-dependence and only numbers allows to use a lot of tricks that cannot be done with generic data. For example, double delta compression, or Gorilla fo
by dima_vm 6y ago
Yes, time-dependence and only numbers allows to use a lot of tricks that cannot be done with generic data.
For example, double delta compression, or Gorilla for floating point numbers. For more, take a look at our open source VictoriaMetrics database, which uses all of such tricks.
- sharpy 6y agoYep. The nature of the time series data does allow for storage optimizations like that. But there is the usage pattern aspect as well, that is more interesting in my opinion. - Availability preferred over consistency (you want your metrics when bad things are happening, even if they may not be 100% accurate) - Extremely write heavy, with most data never being read (they mentioned that only 2% of data is ever read. in my experience at another large company, it was way less than 2%) - For the data that is read, most of reads are for most recent points (mostly alarms, but some dashboards as well) - Different SLAs for queries based on usage - ie. queries for alarms must be fast. Dashboards, and trend analysis - not so much.
- citrin_ru 6y agoClickHouse DBMS allows to combine delta, double delta, Gorilla with LZ4 or ZSTD for column compression. But it is not often used as DB for monitoring metrics so something else is probably expected from time series DB.
- redis_mlc 6y agoIt would work, but Clickhouse is a Russian (Yandex) thing, and SQL isn't really needed for most monitoring and alerting use cases. If I didn't want to use MySQL or Postgres, I'd rank Prometheus #1 and Clickhouse #2. The killer thing for Clickhouse is that Percona supports it, so if you want to outsource the installation, mgmt. and support, you can just write a check and get good results. Also, Clickhouse is a column store with SQL, so you could use an instance for monitoring and another to replace Vertica or Greenplum or whatever so long as it has the client libraries you need.
- rcatcher 6y agoClickHouse is licensed under Apache License 2.0 and Yandex is incorporated in the Netherlands. What are your concerns with it being developed by russians (other than xenophobia)?
- redis_mlc 6y agoNo, you're the xenophobe. I'm concerned with there being a support issue, as well as a smaller user base also affecting support in the long term.