I've been scripting stuff on the pipeline for over a decade and haven't really run into this much. You can define a field separator in the command line with th

I've been scripting stuff on the pipeline for over a decade and haven't really run into this much. You can define a field separator in the command line with the environment variable IFS - i.e. 'IFS=$(echo -en "\n\b");' for newlines - which takes care of the basic cases like spaces in filenames/directory names when doing a for loop, and if I have other highly structured data that is heavily quoted or has some other sort of structure to it, then I either normalize it in some fashion or, as you suggest, write a perl script. I haven't found it too much of a burden, even when dealing with exceptionally large files.