5 ms·

Levels are a great abstraction for debug logs. Debug logs are often abused for much more than their reasonable purpose. Audit logs are different from debug log

Levels are a great abstraction for debug logs. Debug logs are often abused for much more than their reasonable purpose. Audit logs are different from debug logs, and it's pretty common for people to not think about the reasons why. Audit logs are records, and they don't have levels; they might have metadata, but you need to capture all of them, reliably. Debug logs are ephemeral and if you miss some of them you shouldn't care. The problem with structured logging libraries is that they try to do both. There's some in common, but the mechanisms of output should be really different - Debug logs in a circular buffer in RAM makes a lot of sense, but you can't do that for Audit logs. The current approach seems to be "Write them all to the same place, then filter them out", and that only adds more latency in which to lose your audit logs.

tptacek 4y ago
This appears to be why there's a separation of concerns between "handlers" and "loggers".
- GauntletWizard 4y ago
  That's exactly what I'm railing against. You shouldn't be pushing all these things into the same stream and then separating them out again. You may want to make it easy to generate a debug log when you generate an alert. You shouldn't do that by writing a log. You should have a very different set of parameters for If you're a bank, your login "Logs" shouldn't just be an annotated web request; you should have an explicit "AuthenticationSuccess" type in a non-ephemeral data store, and it should contain more than just "301 /auth 1.2.3.4", it should include information about the factors used to make that decision. Semi-structured logs don't have that, but will be abused to provide that. It should be a type system error to create an AuthSuccess without all that, not a set of default parameters to a generic logging call. Structured logging is great for moving fast, and I use it all the time to make it easy to filter through my debug logs, but levels are an abstraction in debug logs - and handlers are completely an anti-pattern
  - tptacek 4y ago
    You've lost me, because the point of having a handler interface is exactly to be able to do things like ensuring that your "authentication success" events get stored differently than your debug messages. You can see upthread that the trend is kind of in exactly the opposite direction to what you're advocating for: plenty of people think there's no such thing as a log or an audit record, only "events", expressed as otel spans. Have fun with those people! :)
marcus_holmes 4y ago
When I last discussed this we ended up in a huge debate about whether we should be sending different log levels to different destinations, because of this dichotomy. Debug messages shouldn't be stored on permanent logs (they frequently contain sensitive/private info). Alerts need to be sent to someone immediately (even if that's a different process monitoring the log file for alerts and raising them). Errors need context in order to diagnose them (ideally a stack trace). All of these are doing different things using the same interface. Log levels end up being abused because of this. What we want is to be able to handle different situations differently. What we get is log levels.
jen20 4y ago
> Audit logs are different from debug logs Very much so - audit logs, where they are important, are often part of the actual business domain and should be explicitly modelled, not delegated to infrastructure with a slightly different level.

tptacek 4y ago

This appears to be why there's a separation of concerns between "handlers" and "loggers".

GauntletWizard 4y ago
That's exactly what I'm railing against. You shouldn't be pushing all these things into the same stream and then separating them out again. You may want to make it easy to generate a debug log when you generate an alert. You shouldn't do that by writing a log. You should have a very different set of parameters for If you're a bank, your login "Logs" shouldn't just be an annotated web request; you should have an explicit "AuthenticationSuccess" type in a non-ephemeral data store, and it should contain more than just "301 /auth 1.2.3.4", it should include information about the factors used to make that decision. Semi-structured logs don't have that, but will be abused to provide that. It should be a type system error to create an AuthSuccess without all that, not a set of default parameters to a generic logging call. Structured logging is great for moving fast, and I use it all the time to make it easy to filter through my debug logs, but levels are an abstraction in debug logs - and handlers are completely an anti-pattern
- tptacek 4y ago
  You've lost me, because the point of having a handler interface is exactly to be able to do things like ensuring that your "authentication success" events get stored differently than your debug messages. You can see upthread that the trend is kind of in exactly the opposite direction to what you're advocating for: plenty of people think there's no such thing as a log or an audit record, only "events", expressed as otel spans. Have fun with those people! :)

marcus_holmes 4y ago

When I last discussed this we ended up in a huge debate about whether we should be sending different log levels to different destinations, because of this dichotomy. Debug messages shouldn't be stored on permanent logs (they frequently contain sensitive/private info). Alerts need to be sent to someone immediately (even if that's a different process monitoring the log file for alerts and raising them). Errors need context in order to diagnose them (ideally a stack trace). All of these are doing different things using the same interface. Log levels end up being abused because of this. What we want is to be able to handle different situations differently. What we get is log levels.

jen20 4y ago

> Audit logs are different from debug logs Very much so - audit logs, where they are important, are often part of the actual business domain and should be explicitly modelled, not delegated to infrastructure with a slightly different level.