4 ms·
You can set up dist eval similar to how it was done in borgmon but you gotta do it manually (or maybe write an operator to automate). One of Monarchs core ideas
by dilyevsky 4y ago
You can set up dist eval similar to how it was done in borgmon but you gotta do it manually (or maybe write an operator to automate). One of Monarchs core ideas is to do that behind the scenes for you
- jeffbee 4y agoPrometheus' own docs say that distributed evaluation is "deemed infeasible".
- dilyevsky 4y agoThats just like.. you know.. their opinion, man. https://prometheus.io/docs/prometheus/latest/federation/ https://prometheus.io/docs/prometheus/latest/federation/
- bpicolo 4y agoPrometheus federation isn't distributed evaluation. It "federates" from other nodes onto a single node. > Federation allows a Prometheus server to scrape selected time series from another Prometheus server
- dilyevsky 4y agoCollect directly from shard-level prometheus then aggregate using /federate at another level. That’s how thanos also works afaik
- hiptobecubic 4y agoThis still requires the query to fit on a single node at some point unless you're doing multi-level aggregation. Monarch does that. Being able to push expensive, highly-parallelizable aspects of the query down into the leaves is a massive benefit.
- dilyevsky 4y agoYep, in theory it’s great if you can afford a team of highly qualified engs to run it like google and dealing with huge metrics. In practice I found it was not hard to crash a monarch mixer (at least some years ago) + most orgs won’t be able to afford nor take advantage of it