3 ms·
I've used Datadog but if you need per query tracing from the database logs, I think the only way is to embed a trace ID in a SQL comment (or in an no-op column)
by sa46 6y ago
I've used Datadog but if you need per query tracing from the database logs, I think the only way is to embed a trace ID in a SQL comment (or in an no-op column) like the grandparent showed.
Datadog is tracing the request from the application akin to something like
openTrace("Query Name", queryStr)
db.Query(queryStr)
closeTrace()
The gap with that style of application-level tracing is that the database logs give no indication of where a query came from, hence the need for embedding a comment in the query with the trace ID.
I would love for a better mechanism than SQL comments for distributed tracing all the way to the database.
- theptip 6y agoI guess what I'm getting at is questioning why you need to look at the database logs (beyond what your APM system can pull from the DB's metrics). What are you getting from the DB logs that you can't get elsewhere?
- sa46 6y agoSure, off the top of my head: - Explain plans for slow queries, via auto_explain in Postgres. You could get really fancy here and convert the pieces of an explain plan (Aggregate, ModifyTable, FullTableScan) into proper distributed traces. Hard to tell, but it looks like GCP might offer that. - Various errors logs associated with queries that require more detail than a SQL state code. For example, lock timeouts. - Debugging, especially root cause analysis when you're trying to figure out why something broke. The trace IDs help build up context.