4 ms·

>Here's the thing: I can do anything with a human >readable text log. I can scan it visually and look for >anomalies; I can use my favorite text edito

>Here's the thing: I can do anything with a human >readable text log. I can scan it visually and look for >anomalies; I can use my favorite text editor with it; I >can grep it; I can cobble together some magic >incantation on a command line to munge it to death >until it cries out in mercy and gives me exactly what I'm looking for. That's basically why the article is proposing using JSON: it's easily readable/scannable by a human, can be easily opened in a text editor, isn't all garbagy like XML, has near perfect support in every scripting language and is rumored to be Christ 2.0. >But if I ever open up a server log and get blasted by a jillion >lines of JSON that make it a thousand times harder for me to figure >out how something broke I'm going to go on a shooting spree. Are these really that different?: >host.com:80 w.x.y.z - - [25/Dec/2011:06:07:42 -0600] "POST /page/open HTTP/1.1" >200 1200 "-" "Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101 Firefox/8.0" >{'host':'host.com', 'timestamp':'2011-12-25 06:07:42', 'method': 'POST', >'path':'/page/open', 'protocol':'HTTP 1/1", 'response_code':200, >'content_length':1200, 'user_agent': 'Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101 Firefox/8.0'} There's a bit of extra verbiage, but the JSON version is perfectly extensible and would allow the devs to add additional debugging information to help you find issues. That said, I can see the downside to letting devs insert random crap, because you could get reams of garbage in the logs. As example of possibly helpful info it'd be easy to add db and app times: >{'host':'host.com', 'timestamp':'2011-12-25 06:07:42', 'method': 'POST', 'path':'/page/open', >'db_time':0.157, 'app_time':0.43, 'protocol':'HTTP 1/1", 'response_code':200, >'content_length':1200, 'user_agent': 'Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101 Firefox/8.0'}

thaumaturgy 15y ago
> ...isn't all garbagy like XML... I don't think it's fair to take something that isn't garbagy at all and then say that the proposed alternative isn't bad because it's not as garbagy as one of the garbagiest things ever invented. :-) JSON may not be as garbagy as XML, but it's still garbagy. With logs in their current format, often my very first tool in log analysis is the scroll wheel on my mouse. It's so lovely because all the lines are unbroken, the columns line up so neatly, and the data in the columns are so distinct from each-other. JSON dicks all that up. Now, OK, I'll admit it wouldn't be very hard for me to whip up something that will take a JSON-formatted log and turn it into something I actually want to read. And in fact that's exactly what I'd end up doing. But, logs are already getting big enough that some companies are doing very stupid things to them. (Bluehost, by default, logs only the last 24 hours for all hosted sites; if you turn on their "extended" logs feature, their logs still have huge gaps in them.) JSON makes that problem a lot worse for those companies. And, if their log rotation isn't set up correctly -- and if they're doing vhosting I'll bet you a donut it's not -- then what would have been a silly amount of data to process now becomes nearly impossible to process.
- CoffeeDregs 15y ago
  >> ...isn't all garbagy like XML... >I don't think it's fair to take something that isn't garbagy at all >and then say that the proposed alternative isn't bad because it's not as >garbagy as one of the garbagiest things ever invented. :-) I can see how what I wrote could be interpreted to say "JSON is a little garbagy", but I didn't mean to imply that JSON is at all garbagy. In fact, in comparison to the standard web log, it's very readable. Further, as a dev, I read web logs pretty frequently and I've forgotten what the "- -" was in the log line I pasted... EDIT: hey, I ignored most of your reply and that's rude: My first tool for debugging Apache is 'less', so I would be screaming and yelling if JSON logs were harder to read than non-JSON logs. Agreed about the size of the logs, but JSON logging seems to only change the caliber of the handgun used to shoot yourself in the foot, not turn it into a Howitzer.
spc476 15y ago
RFC-5424 describes a format to log structured data via syslog, so why do we need JSON again? (five years ago, I bet this proposal would have used XML for encoding)

thaumaturgy 15y ago

> ...isn't all garbagy like XML... I don't think it's fair to take something that isn't garbagy at all and then say that the proposed alternative isn't bad because it's not as garbagy as one of the garbagiest things ever invented. :-) JSON may not be as garbagy as XML, but it's still garbagy. With logs in their current format, often my very first tool in log analysis is the scroll wheel on my mouse. It's so lovely because all the lines are unbroken, the columns line up so neatly, and the data in the columns are so distinct from each-other. JSON dicks all that up. Now, OK, I'll admit it wouldn't be very hard for me to whip up something that will take a JSON-formatted log and turn it into something I actually want to read. And in fact that's exactly what I'd end up doing. But, logs are already getting big enough that some companies are doing very stupid things to them. (Bluehost, by default, logs only the last 24 hours for all hosted sites; if you turn on their "extended" logs feature, their logs still have huge gaps in them.) JSON makes that problem a lot worse for those companies. And, if their log rotation isn't set up correctly -- and if they're doing vhosting I'll bet you a donut it's not -- then what would have been a silly amount of data to process now becomes nearly impossible to process.

CoffeeDregs 15y ago
>> ...isn't all garbagy like XML... >I don't think it's fair to take something that isn't garbagy at all >and then say that the proposed alternative isn't bad because it's not as >garbagy as one of the garbagiest things ever invented. :-) I can see how what I wrote could be interpreted to say "JSON is a little garbagy", but I didn't mean to imply that JSON is at all garbagy. In fact, in comparison to the standard web log, it's very readable. Further, as a dev, I read web logs pretty frequently and I've forgotten what the "- -" was in the log line I pasted... EDIT: hey, I ignored most of your reply and that's rude: My first tool for debugging Apache is 'less', so I would be screaming and yelling if JSON logs were harder to read than non-JSON logs. Agreed about the size of the logs, but JSON logging seems to only change the caliber of the handgun used to shoot yourself in the foot, not turn it into a Howitzer.

spc476 15y ago

RFC-5424 describes a format to log structured data via syslog, so why do we need JSON again? (five years ago, I bet this proposal would have used XML for encoding)