4 ms·

Can it do the following? I have a dev server which is spitting lots of logs and I want to interactively search the logs with something like `fzf`. No changes ca

Can it do the following? I have a dev server which is spitting lots of logs and I want to interactively search the logs with something like `fzf`. No changes can be made to the dev server.

kazinator 4y ago
But suppose you're debugging some scenario (e.g. particular kind of request that generates some logs according to its access/interaction pattern). You can drill into this: interactively determine what things to filter for, trigger the display snapshot/update on particular multi-line patterns and such. You can have it so that whenever you trigger the use case, the new instance of the log appears in the display and freezes, and contains just the relevant information.
anotherhue 4y ago
Tail the logs into fzf --no-sort --tac
- gh123man 4y ago
  I keep this alias in my zshrc for exactly this purpose alias lzf="fzf +s --tac --bind 'enter:select-all+accept' -m" With the added bonus that you can press return to release the filtered lines to stdout. Really useful for interactively filtering logs.
kazinator 4y ago
No. pw works strictly with a small window of live data. It doesn't assume that the file descriptor being read is a seekable file. It doesn't hoard all the data for searching. If there is some dynamic pattern in the logs that repeats (or that you can cause to repeat by executing some test steps on the software which produces the logs), you may be able to use pw to latch on to that pattern. E.g. suppose there is detailed function enter/leave tracing in the log. You can discover that certain functions are traced when you do something, and then filter for that material to remove unrelated spewage, and perhaps set up a trigger to capture the scenario you are reproducing. Once you know that, you can use other tools to find the same thing. You know what to look for in off-line analysis of a complete log. This is not just for logs. If you pipe tcpdump -i <eth> -l | pw you basically get an interactive network monitoring utility. E.g. add: :g thishost.*>.*thathost and now you're just seeing packets from thishost to thathost.

Can it do the following? I have a dev server which is spitting lots of logs and I want to interactively search the logs with something like `fzf`. No changes can be made to the dev server.

kazinator 4y ago

But suppose you're debugging some scenario (e.g. particular kind of request that generates some logs according to its access/interaction pattern). You can drill into this: interactively determine what things to filter for, trigger the display snapshot/update on particular multi-line patterns and such. You can have it so that whenever you trigger the use case, the new instance of the log appears in the display and freezes, and contains just the relevant information.

anotherhue 4y ago

Tail the logs into fzf --no-sort --tac

gh123man 4y ago
I keep this alias in my zshrc for exactly this purpose alias lzf="fzf +s --tac --bind 'enter:select-all+accept' -m" With the added bonus that you can press return to release the filtered lines to stdout. Really useful for interactively filtering logs.

kazinator 4y ago

No. pw works strictly with a small window of live data. It doesn't assume that the file descriptor being read is a seekable file. It doesn't hoard all the data for searching. If there is some dynamic pattern in the logs that repeats (or that you can cause to repeat by executing some test steps on the software which produces the logs), you may be able to use pw to latch on to that pattern. E.g. suppose there is detailed function enter/leave tracing in the log. You can discover that certain functions are traced when you do something, and then filter for that material to remove unrelated spewage, and perhaps set up a trigger to capture the scenario you are reproducing. Once you know that, you can use other tools to find the same thing. You know what to look for in off-line analysis of a complete log. This is not just for logs. If you pipe tcpdump -i <eth> -l | pw you basically get an interactive network monitoring utility. E.g. add: :g thishost.*>.*thathost and now you're just seeing packets from thishost to thathost.