valyala

26 ms·

361.
by valyala 4y ago
VictoriaMetrics author here. I'm working on VictoriaLogs right now, e.g. the logging system on top of VictoriaMetrics architecture ideas. Preliminary results are promising: - It will need much lower amounts of disk space, disk IO, CPU
362.
by valyala 4y ago
If you are looking for a good replacement for InfluxDB, then take a look at VictoriaMetrics. It accepts data in Influx line protocol [1] and it needs less RAM and disk space comparing to InfluxDB [2]. [1] https://docs.victoriamet
363.
VictoriaMetrics: Scaling to 100M Metrics per Second(slideshare.net)
1 points by valyala 4y ago|0 comments
364.
by valyala 4y ago
VictoriaMetrics core developer here. Intetesting paper! It mentions VictoriaMetrics, but doesn't compare its write and read performance and scalability to Mach. It would be great performing such a comparison, since VictoriaMetrics shou
365.
by valyala 4y ago
VictoriaMetrics is based on ClickHouse ideas, but is specifically optimized for storing and querying floating-point time series with arbitrary sets of (key=value) tags. Such time series are also known as metrics or measurements. See [1], [2
366.
by valyala 4y ago
Agreed with this. That's why we at VictoriaMetrics are investigating a hybrid storage scheme - to store recently added data at block storage, while gradually moving older data from block storage to object storage in background. On the
367.
by valyala 4y ago
Both VictoriaMetrics and Grafana Mimir perfectly fit for long-term storage for Prometheus data. The difference is in the used data storage types - VictoriaMetrics stores data to persistent disks (aka block storage), while Grafana Mimir stor
368.
by valyala 4y ago
VictoriaMetrics core developer here. Benchmarks are non-trivial to run properly. Especially when conducting a benchmark against one of the competitor. We tried hard to create a benchmark based on production-like data. It collects data from
369.
by valyala 4y ago
ClickHouse [1] is a "fast open source (almost)SQL analytics database" you are looking for :) [1] https://clickhouse.com/
370.
by valyala 5y ago
Mimir needs 30TB of RAM for a billion of active time series. That's too big price for horizontal scalability comparing to the original Prometheus, which requires around 4TB of RAM for a billion of active time series according to https
371.
by valyala 5y ago
> Which restrictions do you have in mind? You wrote in the previous comment: > ... so external systems (eg Prometheus) can read data from them easily. I pointed to an issue, which prevents from practical usage of remote read protocol
372.
by valyala 5y ago
> I can't find any other open source time series database except Mimir/Cortex which allows this much scale (clustering options in their open source version) The following open source time series databases also can scale horizo
373.
by valyala 5y ago
It would be great if you could provide a few practical examples for "Prometheus remote-read" protocol given its' restrictions [1]. [1] https://github.com/prometheus/prometheus/issues/4456
374.
by valyala 5y ago
Go applications can be built into static self-contained binaries, which don't need extra dependencies. These binaries can run inside `scratch` Docker container, which doesn't take additional space. So it is easy to create small Do
375.
by valyala 5y ago
At first, let's give the definition of `time series`. This is a series of (timestamp, value) pairs ordered by timestamp. The `value` may contain arbitrary data - a floating-point value, a text, a json, a data structure with many column
376.
by valyala 5y ago
FYI, ClickHouse supports querying PostgreSQL data [1], so it is quite easy to mix both analytical data from ClickHouse and relational data from Postgres in a single query. [1] https://clickhouse.com/docs/en/engines
377.
by valyala 5y ago
Time series databases are actually a subset of OLAP databases. The main difference between time series databases and OLTP databases is the amounts of data stored and processed. While OLTP databases can process billions of rows per node, tim
378.
by valyala 6y ago
3) comparison with ClickHouse 4) comparison with VictoriaMetrics
379.
by valyala 6y ago
> Does anyone have any quick checklist or metrics to make decisions like this? When does it make sense to evolve from a vanilla RDB to a timeseries one? 10 daily metrics for 350 entities result in `10 * 350=3.5K` reading per day. This tr
380.
by valyala 6y ago
Do you work on CMS project at CERN? As I know, they successfully use VictoriaMetrics for collecting and analyzing monitoring data. See https://indico.cern.ch/event/877333/contributions/3696707/at... and
381.
by valyala 6y ago
ClickHouse is great OLAP database with outstanding performance! It can be used for collecting and querying observability data [1]. But it may be hard to properly design database schema for ClickHouse for storing general-purpose observabilit
382.
by valyala 6y ago
See also VictoriaMetrics vs TimescaleDB vs InfluxDB benchmarks: - https://valyala.medium.com/measuring-vertical-scalability-fo... - https://valyala.medium.com/when-size-matters-benchmarking-vi... - https:&
383.
by valyala 6y ago
What about comparing TimescaleDB to VictoriaMetrics? There are some old benchmarks [1], but it would be great to see updated benchmarks as well for the latest TimescaleDB and VictoriaMetrics versions. [1] https://valyala.medium.c
384.
by valyala 6y ago
Horizontally scalable systems are usually slower than single-node solutions because of overhead on passing data between nodes. We at VictoriaMetrics clearly see this - cluster version of VictoriaMetrics needs up to 2x more CPU and RAM resou
385.
by valyala 6y ago
This should be already possible - just collect Influx line protocol data with vmagent [1] and write it to Promscale. [1] https://victoriametrics.github.io/vmagent.html
386.
by valyala 6y ago
It would be great conducting a benchmark between Promscale and VictoriaMetrics similar to this one - https://valyala.medium.com/prometheus-vs-victoriametrics-ben... . It differs from TSBS in the following aspects: * It is b
387.
by valyala 6y ago
VictoriaMetrics CTO here. These tests are great, because they spotted a few minor bugs in our PromQL implementation and these bugs were fixed quickly. See https://victoriametrics.github.io/CHANGELOG.html . The majority of f
388.
by valyala 6y ago
VictoriaMetrics CTO here. We were very happy to help Percona integrating VictoriaMetrics in PMM. Some interesting details: * Prometheus has been completely replaced by VictoriaMetrics at PMM server. The end result is lower disk space usage
389.
by valyala 6y ago
This benchmark is much closer to production cases than other benchmarks such as TSBS, because it uses real metrics from node_exporter under load.
390.
Prometheus vs. VictoriaMetrics benchmark on node_exporter metrics(valyala.medium.com)
1 points by valyala 6y ago|1 comments

More ›