VictoriaMetrics core developer here. Benchmarks are non-trivial to run properly. Especially when conducting a benchmark against one of the competitor. We tried

VictoriaMetrics core developer here. Benchmarks are non-trivial to run properly. Especially when conducting a benchmark against one of the competitor. We tried hard to create a benchmark based on production-like data. It collects data from the most frequently monitored scrape target in Prometheus ecosystem - node_exporter. At the same time the benchmark runs queries based on real-world alerting rules for the metrics collected from node_exporter. Resource usage (cpu, ram, disk io, disk space usage) is recorded during the benchmark, so later these metrics can be analyzed and compared. The benchmark runs for 24 hours, so it catches all the transitional states during this duration such as periodic data compactions. We hope the benchmark feamework will be re-used by others for comparing the performance and resource usage for Prometheus-like monitoring systems such as Cortex, M3DB, Thanos, Promscale, etc.