3 ms·

> If you've got good basic infrastructure [...] We do have good basic infrastructure, thanks. We use Puppet and have a decent deploy system that performs atomi

> If you've got good basic infrastructure [...] We do have good basic infrastructure, thanks. We use Puppet and have a decent deploy system that performs atomic deploys from Git. We also do think in terms of services. But the exporter has to run somewhere. About half of our exporters are machine-specific (reads local stats from files or proc or whatever), about half run on the Prometheus node itself and talk to services like ElasticSearch or Postgres. The problem is operational overhead of maintaining a dozen daemons per box, each with its own allocated port. It's not rocket science, just annoying. What could be a small script becomes something unnecessarily big. That's why we're sticking to "textfile" for now. When I have time, my plan is to write a small HTTP server that spawns plugin as subprocesses that emit their metrics via stdout, which seems like a much more reasonable, low-maintenance solution, and something node_exporter ought to support in the first place.

bbrazil 11y ago
> (reads local stats from files or proc or whatever) If there's useful stats from /proc we're missing, we accept PRs. > about half run on the Prometheus node itself and talk to services like ElasticSearch or Postgres. That doesn't sound right, those two should run on the ES/Postgres nodes. Only things like the blackbox exporter and snmp exporter should be considered for running beside Prometheus. > When I have time, my plan is to write a small HTTP server that spawns plugin as subprocesses that emit their metrics via stdout, which seems like a much more reasonable, low-maintenance solution, and something node_exporter ought to support in the first place. We went with the textfile collector approach, rather than reinventing cron.
- lobster_johnson 11y ago
  Sorry, I misspoke. Only non-machine-specific collectors run on the Prometheus machine. > rather than reinventing cron. But Prometheus already has a job interval setting. With text files emitted with cron jobs, you get the silly effect where metrics are produced at intervals which don't correspond with the job interval.

bbrazil 11y ago

> (reads local stats from files or proc or whatever) If there's useful stats from /proc we're missing, we accept PRs. > about half run on the Prometheus node itself and talk to services like ElasticSearch or Postgres. That doesn't sound right, those two should run on the ES/Postgres nodes. Only things like the blackbox exporter and snmp exporter should be considered for running beside Prometheus. > When I have time, my plan is to write a small HTTP server that spawns plugin as subprocesses that emit their metrics via stdout, which seems like a much more reasonable, low-maintenance solution, and something node_exporter ought to support in the first place. We went with the textfile collector approach, rather than reinventing cron.

lobster_johnson 11y ago
Sorry, I misspoke. Only non-machine-specific collectors run on the Prometheus machine. > rather than reinventing cron. But Prometheus already has a job interval setting. With text files emitted with cron jobs, you get the silly effect where metrics are produced at intervals which don't correspond with the job interval.