3 ms·

Separate ingest / compaction / query sounds pretty useful - literally just this morning I needed to learn about docker memory limits, because somehow querying 1

Separate ingest / compaction / query sounds pretty useful - literally just this morning I needed to learn about docker memory limits, because somehow querying 10 days of data OOM’ed my 32GB server… (And my entire database, holding a little over a year of data, is only 33GB on disk o_O) (Now it’s running in a container, and the container crashes and restarts whenever somebody opens a medium-sized dashboard, causing a few seconds of lost metrics - so being able to OOM-kill the query-daemon while leaving the ingest-daemon running sounds like a step forwards :) )

valyala 3y ago
Separating the storage layer from data ingestion and query layer is very smart idea, which is used behind VictoriaMetrics cluster architecture [1]. It makes the cluster more reliable when either data ingestion or query side has temporary issues. This architecture also allows independent scaling of data ingestion, query and storage layers. [1] https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview https://docs.victoriametrics.com/Cluster-VictoriaMetrics.htm...