4 ms·

VictoriaLogs core developer here. The main motivation for creating VictoriaLogs was not so good experience with the existing open source solutions for logs - m

VictoriaLogs core developer here. The main motivation for creating VictoriaLogs was not so good experience with the existing open source solutions for logs - mainly ElasticSearch (or Opensearch) and Grafana Loki. These systems have the following annoying issues, which make users unhappy: - Non-trivial setup and operation - Hard to use query language - Too high resource usage (CPU, RAM, disk space, disk IO) - Bad integration with the traditional command-line tools for logs analysis - grep, head, less, jq, awk, etc. So we at VictoriaMetrics decided creating user-friendly open source database for logs - VictoriaLogs. We believe it solves pain points of Elasticsearch and Grafana Loki mentioned above. Try it on your production workloads and compare how it feels to your current solution for logs. https://docs.victoriametrics.com/VictoriaLogs/ https://docs.victoriametrics.com/VictoriaLogs/

mdaniel 3y ago
Since you're here, I would really enjoy reading about the operational concerns about running V.L. for real (and my apologies for not digging into the repo for "code is the truth" type docs): - Does it use its own flatfile system, or some novel storage mechanism? - Any plans on offering "S3-ish" store for practically unlimited storage (at, of course, the risk of latency for fetches)? - Does it tolerate running on NFS?
- valyala 3y ago
  - VictoriaLogs uses its own storage format, which is inspired by ClickHouse storage format [1]. This is basically a column storage format optimized for structured logs with arbitrary number of fields. - Yes, VictoriaLogs is going to support object storage. See our roadmap [2] - VictoriaLogs should work good on NFS. It writes data to disk in big sequential chunks, which should work optimally on NFS. It reads data in parallel from multiple threads during queries. This should help increasing query performance on high-latency storage such as NFS. [1] https://clickhouse.com/docs/en/development/architecture https://clickhouse.com/docs/en/development/architecture [2] https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html

mdaniel 3y ago

Since you're here, I would really enjoy reading about the operational concerns about running V.L. for real (and my apologies for not digging into the repo for "code is the truth" type docs): - Does it use its own flatfile system, or some novel storage mechanism? - Any plans on offering "S3-ish" store for practically unlimited storage (at, of course, the risk of latency for fetches)? - Does it tolerate running on NFS?

valyala 3y ago
- VictoriaLogs uses its own storage format, which is inspired by ClickHouse storage format [1]. This is basically a column storage format optimized for structured logs with arbitrary number of fields. - Yes, VictoriaLogs is going to support object storage. See our roadmap [2] - VictoriaLogs should work good on NFS. It writes data to disk in big sequential chunks, which should work optimally on NFS. It reads data in parallel from multiple threads during queries. This should help increasing query performance on high-latency storage such as NFS. [1] https://clickhouse.com/docs/en/development/architecture https://clickhouse.com/docs/en/development/architecture [2] https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html