I have a ton of custom log monitoring written in awk/sed because it's way faster to process it that way in shell scripts than it would be in python. The key thi

I have a ton of custom log monitoring written in awk/sed because it's way faster to process it that way in shell scripts than it would be in python. The key thing is that it doesn't need to allocate any memory and that it's by default processing text streams, which means by default you're just zipping the log files through the processor cache and outputting the result to a file to get emailed off. It's important to realize that when you have gigabytes of data you need to analyze that you can go really fast this way. Python sure does have stuff like StringIO, but it just doesn't feel like the right kind of tool for that job. A ton of stuff is just columnar data like CSV or log files out there, and people have been using shell tools to process them for decades. If you're having a hard time reading your complicated awk one liners, I suggest pretty printing it. [1] Plenty of people are using this in production by writing scripts in files. [1] https://stackoverflow.com/questions/55745956/is-it-possible-to-pretty-print-awks-code https://stackoverflow.com/questions/55745956/is-it-possible-...