3 ms·
It's a topic where people default to doing more or less counter productive things because they are not clear on goals. Basically, if you have some kind of clou
by jillesvangurp 6y ago
It's a topic where people default to doing more or less counter productive things because they are not clear on goals.
Basically, if you have some kind of cloud based server infrastructure and you do logging, the goal is to get it aggregated in a place where you can do something productive with it. We are talking about a time series database where it is all about the metadata and less about the message. The message is still important but only if you can query it out of millions of other and possibly very similar messages without too much hassle.
Fallacy #1 is believing that a log event is a string. It's not. You might believe it is a string and format it as a string and you might even have some semi structured way of doing that and tools that feed your confirmation bias by treating logs as lines of text. But in the end you something is going to be picking that string apart to extract some meta data just so it can be inserted in a database and aggregated to the right place. Half the success is not losing the data that matters through wacky formatting, using the wrong tools for the job, etc.
Of course this picking apart if strings is a lot easier if you use some kind of standardized log event format. Which one is appropriate probably depends on where you aggregate your logs. The trick here is avoiding building assumptions about this into your software and avoiding prematurely discarding information that upstream aggregations might benefit from. A lot of stuff happens in between where you invoke some info or warn function in your code and where a message gets stored in some log event aggregator.
Some of the better logging frameworks typically have these features: log levels, an API for logging, an external way of configuring the output of the log engine (formatting, coloring, filtering things out), and some notion of attaching meta data to the events.
The latter is one of those things that are super useful but also one that relatively few developers even know is possible. If you are unfamiliar with the notion of a Mapped Diagnostic Context (MDC): this is where you put any structured information related to sessions, thread ids, and other context. Preserving this information all the way to your log aggregator is the goal.
- ChrisFoster 6y agoAgreed, treating log events as a string seems like throwing a lot of readily available information away. There's various logging libraries which do allow more structure but too often this is ignored. I think we got some of this right in the standard Julia logging frontend. We allow the user to attach arbitrary key value pairs (name=data) to the event, and also automatically attach some statically known context such as source location. (The loggers are also dynamically scoped to the task, which has several good properties. For example it gives great control over attaching context known by the caller to the callee's messages and over dynamically routing or filtering messages. Sort of like MDC but more convenient.)
- wsargent 6y agoAgree with everything you've said, but MDC does have one big drawback: it's only convenient when you're working within a single thread. If you're writing asynchronous code (Scala, Akka, Vert.x etc) then MDC doesn't help you very much. [1] I generally find it easier to have an explicit "context" object and pass that around in an operation, and then log once that context has been built up. If you're lucky, the logging framework will do that for you. Bunyan did this the Right Way from the get go, using `log.child`. [2] [1] https://tersesystems.github.io/blindsight/usage/context.html#mapped-diagnostic-context https://tersesystems.github.io/blindsight/usage/context.html... [2] https://github.com/trentm/node-bunyan#logchild https://github.com/trentm/node-bunyan#logchild
- jillesvangurp 6y agoWith Kotlin, you there is a way to make the MDC work with co-routines. Basically, it works by leveraging the co-routine scope to keep track of a context. This ships as part of the co-routines library in the form of extension functions on the slf4j API: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-slf4j/ https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutin...