3 ms·

1. Logs should contain as much information as possible in order to debug. If you have to ask a user to reproduce a bug so that you can insert extra logging stat

1. Logs should contain as much information as possible in order to debug. If you have to ask a user to reproduce a bug so that you can insert extra logging statements then I think you're doing it wrong. You want to be able to go straight to your logs and have enough information to find the problem immediately without further delay. For example, how are you going to debug a syncing error in a file sync app if you're not keeping detailed logs as to scans, inotify events, incoming and outgoing transactions etc.? 2. You almost want your log entries to tell a story, you want to spend nearly as much time on your log entries as on your code. I think the more you get this right, the better your software design will be. 3. You also want to be able to grep your logs easily using a transaction and user ID. For example, if you just fetched a batch of emails over IMAP, you want to have a transaction ID to describe the session, and include this in all related log entries for the session. You also want to include an account ID in all related entries so you can see all fetches for a given user. 4. Your log entries should do as much work for you as possible. For example, if your log entry logs a filename or string that contains unusual Unicode characters, then you might want your log function to also log what Unicode normalization form the string contains (NFC, NFD, Mixed). This can make otherwise complex bugs trivial to solve. 5. Logs should also be distributed, in the sense that you don't ship all your logs to a central location, but rather have each server manage its own logs. If you need to query your logs, then send a query out to the relevant servers and get them to do the work for you. If logs are always a fraction of a server's data output then you can easily manage gigabytes of log data per server this way.

sagichmal 11y ago
> Logs should contain as much information as possible in order to debug. There's a huge operational and engineering cost buried in that statement, which the article is trying to unearth and provide better solutions for. > 5. Logs should also be distributed, in the sense that you don't ship all your logs to a central location, but rather have each server manage its own logs. Though it does give you some nice properties, this is counter to a lot of accepted best practice... is there a good distributed log query system? `dsh` doesn't cut it, for the record :)
- jorangreef 11y ago
  > There's a huge operational and engineering cost buried in that statement, which the article is trying to unearth and provide better solutions for. Sure, there's always an operational and engineering cost to logging. I think the original comment makes that plain.
goldbrick 11y ago
> Logs should also be distributed... Perhaps there was an unstated assumption on your part, but how are my app servers to know which one handles which query? Is there some sort of log coherence system going on? Are you running Hadoop on your app servers?
- jorangreef 11y ago
  Well, it depends on whether you're designing a distributed system or not, what kind of sharding scheme you use if any etc.
  - goldbrick 11y ago
    > distributed You keep on using that word... but seriously, thanks for the non-answer, now I can be confident you don't know what you're talking about.
    - jorangreef 11y ago
      "now I can be confident you don't know what you're talking about." I guess your original comment made clear that you weren't really interested in an answer beyond dealing with "unstated assumption" and that you want to persist in thinking of logging from a centralized point of view. You just wanted to be "confident". But if you're seriously interested in discussing further to see what I'm working on, feel free to drop me an email on joran@ronomon.com and we can setup a Skype face-to-face.
      - goldbrick 11y ago
        Surely you must realize that I'm going to check out ronomon.com, see that it's absolute shit, and conclude that any further time spent barking up your tree is wasted? Instead of playing internet tough guy, why don't you put something on github, or find some constructive way of obtaining validation? You've got an uphill battle if you're really trying to convince people that having a bunch of text files on their app servers is really so fucking great.

sagichmal 11y ago

> Logs should contain as much information as possible in order to debug. There's a huge operational and engineering cost buried in that statement, which the article is trying to unearth and provide better solutions for. > 5. Logs should also be distributed, in the sense that you don't ship all your logs to a central location, but rather have each server manage its own logs. Though it does give you some nice properties, this is counter to a lot of accepted best practice... is there a good distributed log query system? `dsh` doesn't cut it, for the record :)

jorangreef 11y ago
> There's a huge operational and engineering cost buried in that statement, which the article is trying to unearth and provide better solutions for. Sure, there's always an operational and engineering cost to logging. I think the original comment makes that plain.

goldbrick 11y ago

> Logs should also be distributed... Perhaps there was an unstated assumption on your part, but how are my app servers to know which one handles which query? Is there some sort of log coherence system going on? Are you running Hadoop on your app servers?

jorangreef 11y ago
Well, it depends on whether you're designing a distributed system or not, what kind of sharding scheme you use if any etc.
- goldbrick 11y ago
  > distributed You keep on using that word... but seriously, thanks for the non-answer, now I can be confident you don't know what you're talking about.
  - jorangreef 11y ago
    "now I can be confident you don't know what you're talking about." I guess your original comment made clear that you weren't really interested in an answer beyond dealing with "unstated assumption" and that you want to persist in thinking of logging from a centralized point of view. You just wanted to be "confident". But if you're seriously interested in discussing further to see what I'm working on, feel free to drop me an email on joran@ronomon.com and we can setup a Skype face-to-face.
    - goldbrick 11y ago
      Surely you must realize that I'm going to check out ronomon.com, see that it's absolute shit, and conclude that any further time spent barking up your tree is wasted? Instead of playing internet tough guy, why don't you put something on github, or find some constructive way of obtaining validation? You've got an uphill battle if you're really trying to convince people that having a bunch of text files on their app servers is really so fucking great.