3 ms·
It's hard to tell exactly how this works but judging from the tutorial's docker-compose.yml [0] it looks like this runs as a separate API next to Prometheus and
by eatonphil 5y ago
It's hard to tell exactly how this works but judging from the tutorial's docker-compose.yml [0] it looks like this runs as a separate API next to Prometheus and you tell Prometheus to write [1] to Mimir. I'm unclear how reads work from it or maybe there is no read?
Maybe I'm completely misunderstanding.
[0] https://github.com/grafana/mimir/blob/main/docs/sources/tutorials/play-with-grafana-mimir/docker-compose.yml https://github.com/grafana/mimir/blob/main/docs/sources/tuto...
[1] https://github.com/grafana/mimir/blob/main/docs/sources/tutorials/play-with-grafana-mimir/config/prometheus.yaml#L23 https://github.com/grafana/mimir/blob/main/docs/sources/tuto...
- bboreham 5y agoIt’s a centralised multi-tenant store, supporting the Prometheus query API. So you can point clients directly at Mimir, they send in PromQL and they get data back in Json. (Note I work on Mimir)
- eatonphil 5y agoIs there an example of running mimir without prometheus?
- bboreham 5y agoFor example sending metrics from an OpenTelemetry pipeline. Mimir accepts the Prometheus remote-write api, which is protobuf-over-http; can be generated by anything really.
- k8sToGo 5y agoBut who does the scraping of the prometheus agents? Mimir or still prometheus server?
- bboreham 5y agoIf you have systems exporting metrics in Prometheus style, then you can use Prometheus to scrape them and remote-write to Mimir. You can alternately use Prometheus Agent, to save storing the data and running a query engine at the leaf. You can also use the OpenTelemetry suite to perform the same operation, though this is more appealing if you want some other OpenTelemetry features at the same time. Eg if you prefer the ‘pipeline’ style.
- inkel 5y agoYou configure with Remote Write [1] to the Mimir instance. Then the Prometheus agents will send the metrics to Mimir. 1: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write https://prometheus.io/docs/prometheus/latest/configuration/c...
- Duologic 5y agoLast year I wrote a blog post about this exact question: Who watches the watchers? The general takeaway is that you run a minimal prometheus/alertmanager setup that only scrapes the agents, then use a dead man switch-like system to ensure this pipeline keeps working. Link: https://grafana.com/blog/2021/04/08/how-we-use-metamonitoring-prometheus-servers-to-monitor-all-other-prometheus-servers-at-grafana-labs/ https://grafana.com/blog/2021/04/08/how-we-use-metamonitorin...
- pracucci 5y agoMimir exposes both remote write API and Prometheus compatible API. The typical setup is that you configure Prometheus (or Grafana Agent) to remote write to Mimir and then you configure Grafana (or your preferred query tool) to query metrics from Mimir. You may also be interested into looking at a 5 minutes introduction video, where I cover the overall architecture too: https://www.youtube.com/watch?v=ej9y3KILV8g https://www.youtube.com/watch?v=ej9y3KILV8g
- eatonphil 5y agoCool! Personally I don't like watching videos, preferring to read prose or code or see an arch diagram. But good that it's available.
- pracucci 5y agoI'm the author of the video, but personally I also prefer to read prose instead of watching videos! The architecture is covered here: https://grafana.com/docs/mimir/latest/operators-guide/architecture/ https://grafana.com/docs/mimir/latest/operators-guide/archit... There's also an hands-on tutorial here: https://grafana.com/tutorials/play-with-grafana-mimir/ https://grafana.com/tutorials/play-with-grafana-mimir/