3 ms·
At my last employer, I built a filter program, creatively called CSVTools[0], to do something like this. One piece of the project parses CSVs and replaces the c
by MrDOS 6y ago
At my last employer, I built a filter program, creatively called CSVTools[0], to do something like this. One piece of the project parses CSVs and replaces the commas/newlines (in an escaping- and multiline-aware manner, of course) with ASCII record/unit separator characters[1] (0x1E and 0x1F); the other piece converts that format back into well-formed CSV files. I usually used this with GNU awk, and reconfigured RS[2] and FS[3] appropriately. Or you can just set the input separators (IRS/IFS) and produce plaintext output from AWK.
[0]: https://bitbucket.org/rbr/csvtools https://bitbucket.org/rbr/csvtools
[1]: https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text
[2]: https://www.gnu.org/software/gawk/manual/html_node/awk-split-records.html https://www.gnu.org/software/gawk/manual/html_node/awk-split...
[3]: https://www.gnu.org/software/gawk/manual/html_node/Field-Separators.html https://www.gnu.org/software/gawk/manual/html_node/Field-Sep...
- dbro 6y agoGood idea! Looks similar to something I wrote called csvquote https://github.com/dbro/csvquote https://github.com/dbro/csvquote , which enables awk and other command line text tools to work with CSV data that contains embedded commas and newlines.