8 ms·
I get the idea, but then how would you decide when to enable tracing and in which area? IMO, the advantage of logging is that it some degree of information abo
by xg15 5y ago
I get the idea, but then how would you decide when to enable tracing and in which area?
IMO, the advantage of logging is that it some degree of information about actual failure events - in contrast, monitoring only gives you aggregated information and tracing is too expensive to be applied everywhere at any time.
- andrewstuart2 5y agoTracing should be sampled at a rate that makes sense for the dollar impact of an error in the service, versus the dollar cost of your tracing system. $thousands per transaction? Sample 100%. Cheaper to refund a customer than debug the problem every time? Then sample. You'll have plenty of sampled traces with the bug/error shown as soon as the problem is expensive enough to matter. Metrics are great for demonstrating a problem exists (and alerting), and tracing is great for helping you understand how and why it happened once it's at a big enough scale to matter. Beyond that, with a lot of what's coming out of Open Telemetry, you can get fancier like sampling a higher percent of traces to a localized processor middleware that then filters out any traces you doesn't care about (e.g. dropping non-errors or low-quantile latencies).
- xg15 5y agoYeah, but once again, this requires you to already know there is an error, have a dollar impact calculated, etc. There is a lot of talk in this thread about the performance impact of logging. I don't see how sampling 100% wouldn't have a far greater performance impact.