4 ms·
AWK is a great tool, but I found it lacking when working with CSV files where the separator is found in quoted fields [0]. In this case, European number formatt
by car 4y ago
AWK is a great tool, but I found it lacking when working with CSV files where the separator is found in quoted fields [0]. In this case, European number formatting was the problem, tripping up Sqlite's import.
I resorted to pandas, where the CSV import has parameters for the thousands and decimal separators.
Also see AWK HN post from 2021 [1].
[0] https://earthly.dev/blog/awk-csv/ https://earthly.dev/blog/awk-csv/
[1] https://news.ycombinator.com/item?id=28707463 https://news.ycombinator.com/item?id=28707463
- frogger8 4y agoThis always worked for me https://stackoverflow.com/questions/29642102/how-to-make-awk-ignore-the-field-delimiter-inside-double-quotes https://stackoverflow.com/questions/29642102/how-to-make-awk...
- benhoyt 4y agoThe frawk implementation (not quite POSIX, but extremely fast) and my GoAWK implementation both have an "-i csv" option that puts it in "input mode CSV". More here: https://benhoyt.com/writings/goawk-csv/ https://benhoyt.com/writings/goawk-csv/ $ cat quoted.csv "Smith, Bob",42 $ awk -F, '{ print $1 }' quoted.csv "Smith # you want to print the first field: Smith, Bob $ goawk -i csv '{ print $1 }' quoted.csv Smith, Bob # that's better!
- aidenn0 4y agogawk's FPAT is sufficent for any CSV file without newlines in the fields. I've never run into a CSV with newlines in the fields in the wild.