5 ms·

Rezolus: A high-resolution systems performance telemetry agent

28 points by gregwebs 7y ago

gmuslera 7y ago
I wonder how it compares with InfluxData's telegraf. It seem to gather a subset of the metrics collected by telegraf, but maybe it requires far less resources or is meant for scenarios where very frequent measurements are needed.
- solidasparagus 7y ago
  I've used telegraf to sample at 100Hz for ML workloads. I think one difference is that telegraf pushes each measurement to the DB, while Rezolus summarizes this high granularity data to reduce the amount that needs to be stored.
  - fillest 7y ago
    telegraf supports some summarization https://github.com/influxdata/telegraf#aggregator-plugins https://github.com/influxdata/telegraf#aggregator-plugins
siddontang 7y ago
A similar tool may be https://github.com/cloudflare/ebpf_exporter https://github.com/cloudflare/ebpf_exporter but seems Rezolus uses less resource. The most interesting thing is that Rezolus is written in Rust, we may embed it in our project TiKV and let TiKV can trace itself, no need to through outside tools like perf/bcc.
IceyEC 7y ago
"The resource footprint it takes to run Rezolus at this setting [At 10Hz sampling] is tiny, typically under 15% CPU" It's a bit scary that they consider it OK to spend 15% CPU on all of their servers for telemetry!
- brayniac 7y ago
  author here: that's 15% of a single core, not 15% of a machine
dreamcompiler 7y ago
> according to the Nyquist-Shannon Sampling Theorem, the sampling rate must be at least twice the duration of the shortest burst in order to accurately reflect the intensity of a burst. This statement is a bit sloppy. I think what the author is trying to say is the sampling rate must be such that it samples the shortest burst at least twice. Equivalently you could say that the sampling period should be no longer than half the length of the shortest burst. This would accurately reflect the fundamental frequency of the burst, but if the burst is not sinusoidal you'll get aliasing. The effect of which will be to falsely indicate lower-frequency bursts that aren't actually present.

28 points by gregwebs 7y ago

I wonder how it compares with InfluxData's telegraf. It seem to gather a subset of the metrics collected by telegraf, but maybe it requires far less resources or is meant for scenarios where very frequent measurements are needed.

solidasparagus 7y ago
I've used telegraf to sample at 100Hz for ML workloads. I think one difference is that telegraf pushes each measurement to the DB, while Rezolus summarizes this high granularity data to reduce the amount that needs to be stored.
- fillest 7y ago
  telegraf supports some summarization https://github.com/influxdata/telegraf#aggregator-plugins https://github.com/influxdata/telegraf#aggregator-plugins

siddontang 7y ago

A similar tool may be https://github.com/cloudflare/ebpf_exporter https://github.com/cloudflare/ebpf_exporter but seems Rezolus uses less resource. The most interesting thing is that Rezolus is written in Rust, we may embed it in our project TiKV and let TiKV can trace itself, no need to through outside tools like perf/bcc.

IceyEC 7y ago

"The resource footprint it takes to run Rezolus at this setting [At 10Hz sampling] is tiny, typically under 15% CPU" It's a bit scary that they consider it OK to spend 15% CPU on all of their servers for telemetry!

brayniac 7y ago
author here: that's 15% of a single core, not 15% of a machine

dreamcompiler 7y ago

> according to the Nyquist-Shannon Sampling Theorem, the sampling rate must be at least twice the duration of the shortest burst in order to accurately reflect the intensity of a burst. This statement is a bit sloppy. I think what the author is trying to say is the sampling rate must be such that it samples the shortest burst at least twice. Equivalently you could say that the sampling period should be no longer than half the length of the shortest burst. This would accurately reflect the fundamental frequency of the burst, but if the burst is not sinusoidal you'll get aliasing. The effect of which will be to falsely indicate lower-frequency bursts that aren't actually present.