6 ms·

Want to filter logs at collection time? Well, too bad - your use case is impure and you should just fix everything immediately, even that over which you have no

Want to filter logs at collection time? Well, too bad - your use case is impure and you should just fix everything immediately, even that over which you have no direct control. https://github.com/systemd/systemd/issues/6432 https://github.com/systemd/systemd/issues/6432

Poiesis 5y ago
I realize that the linked thread explains that decreasing the log level wouldn't work because everything else would be affected—but couldn't someone combine decreasing the log level with using a different journal namespace?
dale_glass 5y ago
I actually mostly agree with Poettering here that it's kind of a hack and that the ideal is to collect everything and filter afterwards. You can't get back stuff you exclude by mistake. Programs should be logging because there's some value in the information being sent to the log. If it annoys everyone and serves no purpose, the program needs fixing.
- eximius 5y ago
  The programs needing fixing and supporting filtering are not exclusive options.
- InvertedRhodium 5y ago
  I agree that they _should_ be fixed - but life isn't perfect, and we don't always have a way to fix the issue ourselves. > You can't get back stuff you exclude by mistake. Ok. That's fine - it's my system, my data, and my ass on the line if we lose data by mistake. I don't know see why a developer that knows nothing about my problem domain and the constraints under which I'm operating gets to dictate to me that my choice is wrong.
  - Spivak 5y ago
    Because solving the problem you’re describing means having to do work to implement and maintain a new feature. This kind of framework would put journald on the complexity level of rsyslog when currently journald is purposely very dumb. If you went ahead and implemented thoughtfully and sent a patch there’s a good chance it would get merged. Everyone has a idea about what “I just need basic filtering” and so accommodating anyone but not everyone is a recipe for making even more people mad. Telling people to just disable journald on-disk storage and forward to rsyslog and friends if you need complex filtering and don’t want to pay for double storage is a solution that feels bad but is actually generic enough to be useful.
    - InvertedRhodium 5y ago
      > If you went ahead and implemented thoughtfully and sent a patch there’s a good chance it would get merged. Poettering has made it abundantly clear that he will not accept patches that implement arbitrary filtering. There has been very little engagement on his part with people who have legitimate issues that this would resolve, and appears unwilling to accept that there are use-cases where arbitrarily filtering log data at ingestion time has benefits. Why anyone, myself included, would put the time and effort in to developing a patch that it's clear is ideologically opposed by the gatekeepers of that project is beyond me.

Poiesis 5y ago

I realize that the linked thread explains that decreasing the log level wouldn't work because everything else would be affected—but couldn't someone combine decreasing the log level with using a different journal namespace?

dale_glass 5y ago

I actually mostly agree with Poettering here that it's kind of a hack and that the ideal is to collect everything and filter afterwards. You can't get back stuff you exclude by mistake. Programs should be logging because there's some value in the information being sent to the log. If it annoys everyone and serves no purpose, the program needs fixing.

eximius 5y ago
The programs needing fixing and supporting filtering are not exclusive options.
InvertedRhodium 5y ago
I agree that they _should_ be fixed - but life isn't perfect, and we don't always have a way to fix the issue ourselves. > You can't get back stuff you exclude by mistake. Ok. That's fine - it's my system, my data, and my ass on the line if we lose data by mistake. I don't know see why a developer that knows nothing about my problem domain and the constraints under which I'm operating gets to dictate to me that my choice is wrong.
- Spivak 5y ago
  Because solving the problem you’re describing means having to do work to implement and maintain a new feature. This kind of framework would put journald on the complexity level of rsyslog when currently journald is purposely very dumb. If you went ahead and implemented thoughtfully and sent a patch there’s a good chance it would get merged. Everyone has a idea about what “I just need basic filtering” and so accommodating anyone but not everyone is a recipe for making even more people mad. Telling people to just disable journald on-disk storage and forward to rsyslog and friends if you need complex filtering and don’t want to pay for double storage is a solution that feels bad but is actually generic enough to be useful.
  - InvertedRhodium 5y ago
    > If you went ahead and implemented thoughtfully and sent a patch there’s a good chance it would get merged. Poettering has made it abundantly clear that he will not accept patches that implement arbitrary filtering. There has been very little engagement on his part with people who have legitimate issues that this would resolve, and appears unwilling to accept that there are use-cases where arbitrarily filtering log data at ingestion time has benefits. Why anyone, myself included, would put the time and effort in to developing a patch that it's clear is ideologically opposed by the gatekeepers of that project is beyond me.