4 ms·
The dichotomy is real because nobody asks devs to output properly structured logs, or to work with ops teams to associate particular logs with particular states
by peterwwillis 7y ago
The dichotomy is real because nobody asks devs to output properly structured logs, or to work with ops teams to associate particular logs with particular states. Because of that, metrics are the defacto universal measure of the health of a service. Getting lots of 500s? May be time to flip a circuit breaker. Getting lots of logs of an unknown error? Time to... Send an email to the devs and ask what this means, and wait around for an investigation (meanwhile, metrics have identified the likely culprit)
For many of the services I run, I haven't looked at logs in months, because the metrics tell the story. If service is degraded, I can usually correlate it to another downed service, a network failure, or a recent change. No logs needed.
Correlating metrics with logs is great, and proper distributed tracing is a revelation. But if you had to collect and measure just one thing, it's metrics.