3 ms·
I'd love to have a tool to pipe into that removes low-entropy content automatically, perhaps with machine learning. For example, last week I was grepping throu
by jewel 4y ago
I'd love to have a tool to pipe into that removes low-entropy content automatically, perhaps with machine learning.
For example, last week I was grepping through some security logs looking for anomalous logins. There were a bunch of other security events in the log file. Eventually my command looked like `cat /var/log/auth.log | grep -v Disconnected | grep -v cron:session | grep -v 'Invalid user'` and so forth until I only had the "interesting" lines left.
Maybe the right approach to this tool would be to put the lines into a tree based on the longest substrings.
- xorcist 4y agogrep takes multiple arguments so if you want fewer processes then: grep -v -e Disconnected -e cron:session -e "invalid user" /var/log/auth.log would also suffice (or you can use -E which would enable the usual "Disconnected|invalid|cron").
- pabs3 4y agoAfter watching the demo video for PW, I think it is better than what you were doing, but doesn't have the automation you want.
- kazinator 4y agoIt may be a good way to discover what patterns are worth looking for, and then encoding that into a grep. You need grep if you want to keep everything that matches; no skipping. But the cycle of repeatedly killing "tail -f | grep ..." and then trying different patterns is a little less ergonomic than doing it inside one program. If the input is coming from a pipe directly, trying a different set of grep patterns means killing and restarting the program.