3 ms·

Log files being in binary lets you do things like filter based on specific fields rather than on more complicated string matching. For example, you can get log

by danudey 13y ago

Log files being in binary lets you do things like filter based on specific fields rather than on more complicated string matching. For example, you can get log file messages from postfix's smtp process (as opposed to its smtpd process) with one command (journalctl /usr/lib/postfix/smtp), which you can then cut/sort/grep/awk/etc. You can also follow the logs with journalctl -f and go from there as well. The problem that this solves is the classic 'what log file does this daemon log this error level to on this system', which varies a surprising amount between daemons, versions, distributions, etc. I don't actually care which log file I look at, I just want to know what errors monit is getting. systemd and journalctl provide that handily. I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however.

mixmastamyk 13y ago
That sounds handy, but I wonder if json could provide the same thing.
- bkor 13y ago
  It is binary so you can quickly filter (indexes). E.g. on service, but any field is possible. You can output it in json. systemctl gives you the last lines of service output exactly because of the indexes + binary file. Using json for that just because "OMG binary" seems strange. Json is not readable.
  - deleted 13y ago
    [deleted]
  - mixmastamyk 13y ago
    Postgres can query inside json. To be honest I don't care that much about logging as long as it is saved to disk.
Zancarius 13y ago
I don't mind journalctl in spite of my initial reservations, and as you point out with -f, it's not all that different from tailing a text log. I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me. Using journalctl instead of hunting for the appropriate log files in /var/log seems to offer some degree of platform unity. Maybe it's not the right answer, but it works well enough. My primary complaint is how long it can take on a noisy system to filter a large log, but it's not exceptionally noisome. And most of the time, I just want to follow the log. > I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however. While it probably isn't what you mean as you attach the caveat of task separation, it is possible for syslog-ng to collected logging from systemd's journal. It's a little flaky but it works. :)
- hkhanna 13y ago
  >I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me You are right, of course. Binary logging was not the only reason I ultimately switched. The big reason was that I needed a distribution that did not change so frequently (although I love what Arch is doing). Stability was so important for my use case, though, that I ended up switching to Debian Stable. Although journalctl does work well, I admit, I wonder if the advantages could have been accomplished without the binary file format. So many people have homegrown log parsing tools that need to be re-written to handle a binary file format. Or at least re-written to call journalctl as an intermediary.
  - Zancarius 13y ago
    > The big reason was that I needed a distribution that did not change so frequently Yeah, this is a perfectly valid reason and also why I tend not to use Arch on servers (as much as I would like to otherwise, though...). Both have their use cases, and I love Arch for desktops, but it's not a particularly great match for situations where rapid, near-constant change is unwarranted, unnecessary, or problematic. Arch is also a distro that requires a fair amount of love and care--failure to do so, as a friend of mine learned last year when I had to help him update a year old install, can result in an upgrade path that's nearly impossible. Fortunately, ARM was still up at the time. There certainly is something to be said for stable distros with a strict upgrade path. > I wonder if the advantages could have been accomplished without the binary file format. I don't know. I'll be honest: I can't think of a time when I've actually made use of the filtering features of journalctl and systemd's binary format. Nearly all circumstances that have required me to examine the log are usually done by observing the last few entries or manual searches. To that extent, I'd agree that it bears a certain sense of overkill. That said, it is still possible to feed syslog-ng entries from systemd. So I can't think of a reason to fret much over specific tooling or the likes that expects text logs. IMO, the udev/systemd relationship, repo, etc. is probably a much more valid complaint, especially for distributions that aren't planning on using systemd in the near future as it likely has some impact on developer/maintainer workload. Although from a sysadmin perspective, the benefits of systemd are very real and material, at least in my experience. Service introspection could probably use some work to simplify, so that's another complaint. I can never remember the commands half the time. ;)

by danudey 13y ago

mixmastamyk 13y ago

That sounds handy, but I wonder if json could provide the same thing.

bkor 13y ago
It is binary so you can quickly filter (indexes). E.g. on service, but any field is possible. You can output it in json. systemctl gives you the last lines of service output exactly because of the indexes + binary file. Using json for that just because "OMG binary" seems strange. Json is not readable.
- deleted 13y ago
  [deleted]
- mixmastamyk 13y ago
  Postgres can query inside json. To be honest I don't care that much about logging as long as it is saved to disk.

Zancarius 13y ago

I don't mind journalctl in spite of my initial reservations, and as you point out with -f, it's not all that different from tailing a text log. I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me. Using journalctl instead of hunting for the appropriate log files in /var/log seems to offer some degree of platform unity. Maybe it's not the right answer, but it works well enough. My primary complaint is how long it can take on a noisy system to filter a large log, but it's not exceptionally noisome. And most of the time, I just want to follow the log. > I would agree with an approach to have systemd send log events to e.g. journald (or syslog-ng, or whatever the user wanted) to separate that functionality out, however. While it probably isn't what you mean as you attach the caveat of task separation, it is possible for syslog-ng to collected logging from systemd's journal. It's a little flaky but it works. :)

hkhanna 13y ago
>I do suspect binary logging isn't the only reason hkhanna dumped Arch for Debian. It smells of nitpickery to me You are right, of course. Binary logging was not the only reason I ultimately switched. The big reason was that I needed a distribution that did not change so frequently (although I love what Arch is doing). Stability was so important for my use case, though, that I ended up switching to Debian Stable. Although journalctl does work well, I admit, I wonder if the advantages could have been accomplished without the binary file format. So many people have homegrown log parsing tools that need to be re-written to handle a binary file format. Or at least re-written to call journalctl as an intermediary.
- Zancarius 13y ago
  > The big reason was that I needed a distribution that did not change so frequently Yeah, this is a perfectly valid reason and also why I tend not to use Arch on servers (as much as I would like to otherwise, though...). Both have their use cases, and I love Arch for desktops, but it's not a particularly great match for situations where rapid, near-constant change is unwarranted, unnecessary, or problematic. Arch is also a distro that requires a fair amount of love and care--failure to do so, as a friend of mine learned last year when I had to help him update a year old install, can result in an upgrade path that's nearly impossible. Fortunately, ARM was still up at the time. There certainly is something to be said for stable distros with a strict upgrade path. > I wonder if the advantages could have been accomplished without the binary file format. I don't know. I'll be honest: I can't think of a time when I've actually made use of the filtering features of journalctl and systemd's binary format. Nearly all circumstances that have required me to examine the log are usually done by observing the last few entries or manual searches. To that extent, I'd agree that it bears a certain sense of overkill. That said, it is still possible to feed syslog-ng entries from systemd. So I can't think of a reason to fret much over specific tooling or the likes that expects text logs. IMO, the udev/systemd relationship, repo, etc. is probably a much more valid complaint, especially for distributions that aren't planning on using systemd in the near future as it likely has some impact on developer/maintainer workload. Although from a sysadmin perspective, the benefits of systemd are very real and material, at least in my experience. Service introspection could probably use some work to simplify, so that's another complaint. I can never remember the commands half the time. ;)