3 ms·
I think the point here is to not do unstructured, string-based logging. If logs are valuable, then you should treat them the same way as any other valuable data
by lmm 7y ago
I think the point here is to not do unstructured, string-based logging. If logs are valuable, then you should treat them the same way as any other valuable data: rather than blatting string soup into some datastore and hoping you can somehow index it in the future, write a proper data schema and represent the trace of your program execution in a structured, machine-readable way.
This ends up overlapping a lot with event sourcing: if you're doing event sourcing right, with every substantial business decision recorded as an event and your logic decomposed into lots of fine-grained transformations between event streams, then the event streams can be your logs: each successive event transformation either succeeds or fails, and if it fails then you have the input event and the small piece of code in which the failure occurred, so you shouldn't need any more. If a given transformation gets too complicated to debug, split it into two smaller transformations!
You still need alerting on failures, but again, that's something that's better done in a structured way, with proper stacktraces and business-relevant information.
- devchix 7y agoI could never get structured logging to work, syslog receiver for structured logs is a nightmare to set up. Also, when I start writing logs, I just want to quickly set up something to check my error state and move on. The onus of setting up structured logging output in the code is too much work in the immediate moment. Additionally, structured logging does not solve the problem we think it solves, what is the log trying to say. What do I use as my key-value naming convention? fault_type = major, component_name = X, subcomponent_name = X.aaa desc1 = "some text" desc2 = "fooblat" -- I just made all those things up, the log consumer is still going to need to parse out the KV pairs and decipher what that means and what to do. Meanwhile, we have RFC5424 which does a fair job of defining and standardizing those made-up things.
- lmm 7y agoI meant store logs the same way as any other data - be that an SQL database, Cassandra, or whatever you're favouring for this system. I'd expect your fields to be somewhat business-specific, and reading them works the same as reading any other business data - you write the data schema in one place and generate read and write code from it. key-value pairs aren't perfect and you certainly can (and should) define a more detailed schema than that, but even just a bag of key-value pairs is a lot more structure (and therefore a lot easier to query) than a raw string.
- jujodi 7y agoMaybe, sure seems like the point was "do not log ever"
- mrkeen 7y ago* Use [...] error and business monitorings with alerts * make logging an explicit part of your contracts * Logging should be done right
- lmm 7y agoWell yeah, because when we're storing structured data we don't think of that as "logging". But the point isn't to throw that information away, it's to store it properly.