4 ms·
The real takeaway is that log files invariably tend to become interfaces for something. They often end up being used for monitoring tools, business intelligence
by skrebbel 15y ago
The real takeaway is that log files invariably tend to become interfaces for something. They often end up being used for monitoring tools, business intelligence, system diagnostic tools, system tests, and so on. And they're great for this. But not when they contain sentences like "Opening conection...", which break half those tools the moment someone fixes the typo.
The log strings became an interface. Avoid this. If it's an interface, it has to be specced, and it has to allow for backward compatibility, just like any other interface that crosses component / tool boundaries.
Whether you do the actual data storage with JSON or something else doesn't matter. It's an implementation detail (though I agree that keeping it not only machine-readable, but also human-readable, is probably a good thing).
Design the classes that represent log files, and treat them like they're part of a library API. Don't remove fields. Ideally, use the same classes for writing (from your main software) and parsing the logs (from all that other tooling) and include version information in the parser so that the class interface can be current yet the data can be ancient.
- gbog 15y agoThat exactly the opposite of my understanding of logs, and the reason why I can't agree with the OP. For me logs are a way to store extensive historical data about what happened, in the cheapest and simplest way possible. Logs are a "write a lot and read almost never" kind of storage. For this kind of storage, the simplest way to do it is tab delimited flat files. Logs are for debug or "legal" purposes: a client complains he has lost all his data, your boss comes in the room with fume steaming outside his ears telling you names because you "can't store f* client's F* valuables cleany". Then, using awk or grep kungfu, you come up ten minutes later with the exact millisec when the client did click on "Yes, I am sure I want to delete my data", his IP, his session_id, his browser fingerprint and so on. In case of security audit, you are required to send 10 years of server logs to a trusted third-party (they will not do anything with it, they just want to make sure you have logs). You zip the thing and send it by email to then (thus crashing security auditor's email server). You have done your duty. These are what logs are for. Having JSON or any other format inside just make it more fragile, less versatile, will mess with line-oriented commands and is unnecessary. If you are using your logs regularly to track some business data about your product, you are using logs for the wrong purpose and should consider using something else.
- gizzlon 15y agoYou just email them? No encryption or anything like that? Kind of funny that a security audit is the thing that trigger this .. =/