3 ms·

> The query language can fairly efficiently compute metrics at query time rather than needing everything precomputed I'm having hard time figuring out how quer

> The query language can fairly efficiently compute metrics at query time rather than needing everything precomputed I'm having hard time figuring out how query language might affect exectuion time. Granted, I've never seen borgmon in action, so the first question should probably be: how different its query language is from what Prometheus offers? If it's more on imperative side than declarative one then your point might actually make sense to me as it is. > You can also downsample (eg. Change scrape interval to 5 mins after a week). Does it allow different aggregation methods (avg/sum/max/whatever) for different metrics, like Carbon does?

shereadsthenews 8y ago
I think what he is saying is that it is easier to do something ad hoc in Monarch since it is a global service with unified storage and retrieval. A Googler can join the data of Gmail and Ads easily if they want, whereas combining the borgmon data of those services is either not possible or it involves downloading CSVs and performing the join externally. > Does it allow different aggregation methods (avg/sum/max/whatever) for different metrics, like Carbon does? All of these methods are available[1]. Stackdriver Monitioring is the public face of Monarch. 1: https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.alertPolicies?_ga=2.5326451.-1585613562.1554906215#reducer https://cloud.google.com/monitoring/api/ref_v3/rest/v3/proje...
- puzzle 8y ago
  No, I think it's strictly about where aggregations and other computations occur. You hint at that when you mention CSV files. Borgmon keeps X hours of data in memory and looks up anything older from TSDB. I never understood how to aggregate data from the same service, after the fact, in a reliable fashion. Or if it was possible at all. I believe it was only working when looking at data in memory. That's why people had rules to aggregate data at multiple levels as it was being ingested, then persisting that into TSDB. I don't remember the latter being much more sophisticated than a bunch of files in GFS with the timeseries and some metadata, either. Monarch, I think, is based on Bigtable. It has a richer API. It might even use BT coprocessors to perform computations closer to the where the raw data is stored, rather than in the Monarch frontends. (I haven't watched the public talk, so I'm just going from vague recollection.)
  - shereadsthenews 8y ago
    Borgmon/TSDB was also a bigtable, an enormous one known as 'dumptruck'.
    - puzzle 8y ago
      I am pretty sure that was developed for Monarch. Perhaps later it was adopted by BM to consolidate storage, but still through a dumb API? I know that originally it had a bunch of files in GFS, with a layer of indirection, because there was a tool to fix them.

shereadsthenews 8y ago

I think what he is saying is that it is easier to do something ad hoc in Monarch since it is a global service with unified storage and retrieval. A Googler can join the data of Gmail and Ads easily if they want, whereas combining the borgmon data of those services is either not possible or it involves downloading CSVs and performing the join externally. > Does it allow different aggregation methods (avg/sum/max/whatever) for different metrics, like Carbon does? All of these methods are available[1]. Stackdriver Monitioring is the public face of Monarch. 1: https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.alertPolicies?_ga=2.5326451.-1585613562.1554906215#reducer https://cloud.google.com/monitoring/api/ref_v3/rest/v3/proje...

puzzle 8y ago
No, I think it's strictly about where aggregations and other computations occur. You hint at that when you mention CSV files. Borgmon keeps X hours of data in memory and looks up anything older from TSDB. I never understood how to aggregate data from the same service, after the fact, in a reliable fashion. Or if it was possible at all. I believe it was only working when looking at data in memory. That's why people had rules to aggregate data at multiple levels as it was being ingested, then persisting that into TSDB. I don't remember the latter being much more sophisticated than a bunch of files in GFS with the timeseries and some metadata, either. Monarch, I think, is based on Bigtable. It has a richer API. It might even use BT coprocessors to perform computations closer to the where the raw data is stored, rather than in the Monarch frontends. (I haven't watched the public talk, so I'm just going from vague recollection.)
- shereadsthenews 8y ago
  Borgmon/TSDB was also a bigtable, an enormous one known as 'dumptruck'.
  - puzzle 8y ago
    I am pretty sure that was developed for Monarch. Perhaps later it was adopted by BM to consolidate storage, but still through a dumb API? I know that originally it had a bunch of files in GFS, with a layer of indirection, because there was a tool to fix them.