I work in data processing and I use awk occasionally to work with csv files often gigabytes in size. I join csv files that each have a header with awk '(NR

I work in data processing and I use awk occasionally to work with csv files often gigabytes in size. I join csv files that each have a header with awk '(NR == 1) || (FNR > 1)' *.csv > joined.csv Note this only works if your csv files don't contain new lines. However if they do, I recommend using https://github.com/dbro/csvquote https://github.com/dbro/csvquote to circumvent the issue. Yesterday I used awk as a QA tool. I had to subtract a sum of values in the last column of one csv file from another, and I produced a expr $(tail -n+2 file1.csv | awk -F, '{s+=$(NF)} END {print s}') - $(tail -n+2 file2.csv | awk -F, '{s+=$(NF)} END {print s}') beauty. This allowed me to quickly check whether my computation was correct. Doing same in pandas would require loading both files into RAM and writing more code. However I avoid writing awk programs that are longer than a few lines. I am not too familiar with the development environment of awk, and I stick to either Python or Go (for speed) where I know how to debug, jump to definition, write unit tests and read documentation.