4 ms·

Especially if you use something like a tab delimiter, then you typically don't need to specify the delimiter. What if the fields contain tabs? For every human-

Especially if you use something like a tab delimiter, then you typically don't need to specify the delimiter. What if the fields contain tabs? For every human-readable format that can contain arbitrary user input (nearly all of them) you need some form of escaping (I guess you could do length prefixing in ASCII decimal, but it'd not be pretty either and incompatible with basic tools). But by far the biggest problem is the logging of text messages aimed at a humans, not the delimiting. Regular expressions can help in searching logs in quick-hack jobs, but if you need to parse logs for visualization or reporting, which is very common in organizations, using them is error-prone. After all, you rely on English messages of a certain form. The complexity of that could quickly move from "easy with regexps" to "we need NLP in our log parser!" (never mind security problems with one field leaking into another due to a slightly wrong regexps). The application might change the message to make the message more readable for humans, or even move around fields, and your automated parser breaks. Structured messages, on the other hand, won't change for those concerns as the formatting for humans happens in the back-end. I get a bit annoyed at the "WTF kid, learn UNIX tools" kind of responses here. UNIX tools are one way of doing things, not the holy one perfect way. Tool support is important, but there are also tools for processing JSON, XML, streams available. Shouldn't you use the best tool for the job, not the one you happen to know? (I don't mean that JSON is necessarily the best format in every use-case, but for automated processing every structured format at all trumps arbitrarily delimited and escaped files. You can easily convert between structured formats should the need arise)

burgerbrain 14y ago
> "Shouldn't you use the best tool for the job, not the one you happen to know?" Well, 'best' in this case should be determined by combining a number of factors. Certainly technological superiority should be weighted quite heavily, but in certain circumstances the "Nobody ever got fired for buying IBM" effect is also very important. Of course there is also the "The best tool for the job is the one you have/know." quip, which I don't generally agree with myself... so I guess what I'm saying is that your mileage may vary. (disclaimer: I log with JSON)
- wladimir 14y ago
  But such thinking can block innovation. It's a form of path lock-in. Both UNIX and Windows "gurus" are guilty of this, of seeing their way as the "one true way". Just because it's always been that way. New ideas are not always better, but sometimes they might be. In the longer run "I'm used to this" (on its own) is not a good argument as there will always be new people that are not used to your specific blub, and if they can be more productive or make more reliable/secure systems then eventually you'll be out of the market. See also: https://news.ycombinator.com/item?id=3892410 https://news.ycombinator.com/item?id=3892410
  - burgerbrain 14y ago
    Oh certainly, I fully agree. I think I'm just saying that I think logging is going to be one of those things, for better or worse, that most developers look at and do the mental calculus of "It's a good idea, but do I want to go out on a limb here, with this issue?" In at least some cases all the factors added together just won't work out to it being worth the risk/effort. Basically just the IBM thing. Was IBM always the best choice? No. But even so, it was often the best choice for the individuals making that call. This is the sort of thing that you have to recognize and contend with if you want to introduce change.

burgerbrain 14y ago

> "Shouldn't you use the best tool for the job, not the one you happen to know?" Well, 'best' in this case should be determined by combining a number of factors. Certainly technological superiority should be weighted quite heavily, but in certain circumstances the "Nobody ever got fired for buying IBM" effect is also very important. Of course there is also the "The best tool for the job is the one you have/know." quip, which I don't generally agree with myself... so I guess what I'm saying is that your mileage may vary. (disclaimer: I log with JSON)

wladimir 14y ago
But such thinking can block innovation. It's a form of path lock-in. Both UNIX and Windows "gurus" are guilty of this, of seeing their way as the "one true way". Just because it's always been that way. New ideas are not always better, but sometimes they might be. In the longer run "I'm used to this" (on its own) is not a good argument as there will always be new people that are not used to your specific blub, and if they can be more productive or make more reliable/secure systems then eventually you'll be out of the market. See also: https://news.ycombinator.com/item?id=3892410 https://news.ycombinator.com/item?id=3892410
- burgerbrain 14y ago
  Oh certainly, I fully agree. I think I'm just saying that I think logging is going to be one of those things, for better or worse, that most developers look at and do the mental calculus of "It's a good idea, but do I want to go out on a limb here, with this issue?" In at least some cases all the factors added together just won't work out to it being worth the risk/effort. Basically just the IBM thing. Was IBM always the best choice? No. But even so, it was often the best choice for the individuals making that call. This is the sort of thing that you have to recognize and contend with if you want to introduce change.