Monitoring all the things can be hard. There are a lot of different solutions, that have different trade-offs for different volumes of data, cardinality of dat

Monitoring all the things can be hard. There are a lot of different solutions, that have different trade-offs for different volumes of data, cardinality of data, how out of order / async the data may arrive, whether you need push or pull delivery, etc, etc. These apply to you and only you know the parameters and considerations of your application, and the environment it runs in (perhaps you come with the baggage of "this is instrumented in x and I can't change that" in addition to volume and cardinality considerations, perhaps it's a point-of-sale in a shop that only connects to the internet once a day for upload of the data). But for the simplest solution, you're probably talking Prometheus + Loki/Elastic + Tempo + Pyroscope/Polar Signals + Grafana. This is simple as in "all pieces are proven and work in the small and get you fairly far"... but you still have to compose the solution out of those parts. And later you may wish to graduate to Grafana Cloud when the pain of running it all yourself is taking more time and effort than running the systems you're supposed to be working on. Oh, and if you really want to lean in on traces for everything, take look at Honeycomb too. As to "the monitoring costs more than the app"... yup, at Cloudflare for a very very long time there was little monitoring of DNS requests or billing of it... because the act of counting it and recording it cost more than just serving the request. This principle scales well (or badly if you're looking at your credit card bill)... and is also why those who make these systems really have to keep in mind how to make the instrumentation cost less (eBPF?) as well as the ingestion and retention cost less (various methods, including aggregation, sampling, etc).