4 ms·
How do you even log as JSON? Is your entire log file a giant JSON array? That would be challenging for most parsers I know because they would have to read the
by Hopka 14y ago
How do you even log as JSON?
Is your entire log file a giant JSON array? That would be challenging for most parsers I know because they would have to read the entire array into memory first.
Or do you log one JSON object per line? Then you would get problems as soon as you have line breaks inside strings and still have to parse until the object ends in some other line. Also, JSON objects do not have to be single-line to be valid, so you would in fact be working with some self-defined subset of JSON.
- drostie 14y agoIs your entire log file a giant JSON array? That's not necessarily a bad choice, but it's problematic for the reasons you describe. Still, since JSON is concatenative, you could indeed store all of your objects with a comma at the end and then use: import json; json.loads("[%s null]" % file_contents)[:-1] Then you would get problems as soon as you have line breaks inside strings Then you are not logging JSON. JSON does not permit that. Also, JSON objects do not have to be single-line to be valid, so you would in fact be working with some self-defined subset of JSON. Yes, and? Working in a subset of JSON which forbids newlines as whitespace -- that's still JSON, and it solves your problem elegantly. Do... do you have multiple logging programs, logging to the same file, and one of them wants to insert newlines? Is this a real problem in your dev stack?
- Hopka 14y agoIt is a real problem in someone's dev stack. I have already written code to preprocess such log files before feeding chunks into a real JSON parser. It didn't make my life easier. I'm not against logging as JSON at all, but as pointed out, you have to use a subset that makes parsing the logs easy.