5 ms·

Just look into log4j: mdc/ndc to include context (user/session), include the thread name in your pattern and use an async appender to take care of performance.

by wokkel 4y ago

Just look into log4j: mdc/ndc to include context (user/session), include the thread name in your pattern and use an async appender to take care of performance. All of these things have been solved already.

chrsig 4y ago
> All of these things have been solved already. I mean...yes, that's how I was able to highlight the benefits of having a single log message per request. > Just look into log4j: mdc/ndc to include context (user/session), include the thread name in your pattern and use an async appender to take care of performance. At a glance this looks like it will still produce 1 log line per logging call, versus buffering with the request. adding the thread id just helps stitch them together after the fact. This doesn't get rid of any performance overhead -- it may get it out of the way of a user request, but it's still there under the hood. In the go case, using the runtime/trace library as an example[0], a `context.Context` is passed down the call stack which gets decorated with trace spans/regions/log messages on the way down. At the end of a task, all of the collected context gets written to the active trace log. [0] https://pkg.go.dev/runtime/trace https://pkg.go.dev/runtime/trace
- wokkel 4y ago
  I like my log to include where something happened and in some complex cases having a few thousand lines of logging info per request is not unreasonable. So stuffing al that info into a single log event/line doesn't make sense to me. I recently had the unfortunate situation where two seemingly independant sessions were interfering, here a sequential logfile where things were logged in the order occurring was the only way to correlate them. Even though we do telemetry via events, old school logfiles are a godsend to debug the complex cases.
wsargent 4y ago
MDC/NDC only reliably works when you don't have asynchronous code (Akka/Vert.x/CompletionStage) through your application. As soon as you start using multiple threads, it becomes significantly more complex to keep MDC context carried over from one thread to the next.
- kaba0 4y ago
  Well, that has more to do with reactive programming not playing by the rules. Once Loom arrives with Extent-Local scopes it will solve it very well.

by wokkel 4y ago

chrsig 4y ago

> All of these things have been solved already. I mean...yes, that's how I was able to highlight the benefits of having a single log message per request. > Just look into log4j: mdc/ndc to include context (user/session), include the thread name in your pattern and use an async appender to take care of performance. At a glance this looks like it will still produce 1 log line per logging call, versus buffering with the request. adding the thread id just helps stitch them together after the fact. This doesn't get rid of any performance overhead -- it may get it out of the way of a user request, but it's still there under the hood. In the go case, using the runtime/trace library as an example[0], a `context.Context` is passed down the call stack which gets decorated with trace spans/regions/log messages on the way down. At the end of a task, all of the collected context gets written to the active trace log. [0] https://pkg.go.dev/runtime/trace https://pkg.go.dev/runtime/trace

wokkel 4y ago
I like my log to include where something happened and in some complex cases having a few thousand lines of logging info per request is not unreasonable. So stuffing al that info into a single log event/line doesn't make sense to me. I recently had the unfortunate situation where two seemingly independant sessions were interfering, here a sequential logfile where things were logged in the order occurring was the only way to correlate them. Even though we do telemetry via events, old school logfiles are a godsend to debug the complex cases.

wsargent 4y ago

MDC/NDC only reliably works when you don't have asynchronous code (Akka/Vert.x/CompletionStage) through your application. As soon as you start using multiple threads, it becomes significantly more complex to keep MDC context carried over from one thread to the next.

kaba0 4y ago
Well, that has more to do with reactive programming not playing by the rules. Once Loom arrives with Extent-Local scopes it will solve it very well.