3 ms·
Another thing - Traces (and often metrics for that matter) are often treated as something that is investigated one at a time. There's a lot of tools out there t
by prpl 3y ago
Another thing - Traces (and often metrics for that matter) are often treated as something that is investigated one at a time. There's a lot of tools out there to help visualize a trace. They are not, usually, treated as something that should be analyzed at scale, and the tooling there is not great.
I think PromScale looked promising for some of those problems but it's also a deprecated product.
I think the state of the art is/will be ingesting traces into Iceberg for some form of analysis with whatever execution engine you want (Trino/Spark/etc...). A lot of places are doing this.
OpenTelemetry is clearly the future of monitoring, probably with a lot of lifting from Prometheus and other things.
- zug_zug 3y agoSo what I want is something like Splunk search for traces. I want to be able to write 12-shareable lines of custom-query and immediately get a pretty graph of multidimensional data "What % of users who just registered this month got a 500 more than half the time on this specific path?" and I want it as a pretty graph over time
- phillipcarter 3y agoThat's precisely what tools like Honeycomb and Lightstep offer. But most vendors don't have a traces-first approach to querying data.
- zug_zug 3y agoReally? I haven't used honeycomb specifically, but you're saying it supports an arbitrarily complex query depth, with any number of joins essentially?
- 8note 3y agoI kinda hacked x-ray onto things, but it does give you an overview aggregate for each node I'm not sure if I've really found it useful though. Im sure if I didn't know the service, itd be great, but I already know where to look for whatever problem