4 ms·
There are two types of logging: a) Logs which break the program semantics when missing. That means, the program does not work "as expected" anymore for any dir
by valenterry 6y ago
There are two types of logging:
a) Logs which break the program semantics when missing. That means, the program does not work "as expected" anymore for any directly or indirectly impacted enduser.
b) Logs that have no semantic impact when missing. No one will notice that they are missing, unless there is any other problem. These logs are solely meant to better understand the program operations.
Logs of type A should be tested like everything else. But logs of type B - I don't think I would test them.
While it is not nice to have a failure in production and figuring out only at that time that the logging doesn't work, I think that is even worse to break tests because of logging changes. Logging usually is so easy that it is an all- or nothing thing. I'm not saying that it does not have advantages to test for these kind of logs, but from my experience it is not worth the costs in 99% of the cases.
- geewee 6y agoCompletely agree with this. I've wanted to test type A logs, and often heard people saying that "you should never test logs" No - you should never test most types of logs - sometimes logs, particularly in error situations, are part of the output of a function.
- parhamn 6y agoThis sounds like a nomenclature thing. I’m sure any reasonable developer would agree it makes sense to test expected output of a program that isn’t strictly for debugging purposes (type A mentioned above), it’s just that is usually called output not “logs”.
- zoomablemind 6y ago> ...and often heard people saying that "you should never test logs" It's surprising that it gets labeled as "logs". Don't we test the expected behavior of the _code_ under test? Be it a business or an auxilliary code like logging, it's still code, some function/class/facility...well, any purpose added code. Thus, I would test such logging facility for correctness; otherwise, the next time its output is needed, I want to be sure that it can be trusted. Of course, using ready-made logging frameworks may help shift such burden of testing onto the framework's developer.
- marcosdumay 6y agoI agree, but... > Logging usually is so easy that it is an all- or nothing thing. By all means, have the integration test that verifies that logs are produced at all (on each mechanism, if you support more than one). And it's a bonus if that test makes the deployment (or installation if that's your thing) fail.
- ivanbakel 6y ago> Logs which break the program semantics when missing. What's an example of this kind of logging? I can imagine perhaps legally-mandated audit logs for financial software, but that would be stretching the definition of "semantics" in my mind.
- xboxnolifes 6y agoMaybe I'm misunderstanding the parent, but I interpreted as the same kind of "logging" as the standard output of a CLI application. You expect a grep tool to output result to standard output for the user to use. Or you would expect user-facing errors to occur if an sql client fails to connect to a server.
- rixrax 6y agoBack in the day when everything was written in C/C++ it wasn’t unusual that seemingly well tested software had careless memcpy or sprintf left in the error/log execution path that then exposed that system to remote code execution. Just wanted to throw this in as to discourage completely ignoring testing logging. Of course e.g. good static analysis should somewhat alleviate chance of surprises here.
- sorokod 6y agoI would suggest a third type: c) Logs that when present, "break" (using the term loosely) the application. The example here is when application is logging sensitive data, e.g. PII