4 ms·
People are lazy. The ASCII C0 separators -- FS, GS, RS, US -- are non-printable (by design), but this impacts their usability by laypeople, because they don't
by niftich 9y ago
People are lazy.
The ASCII C0 separators -- FS, GS, RS, US -- are non-printable (by design), but this impacts their usability by laypeople, because they don't show up as obvious symbols in a plaintext editor, and most critically, they visually seem to be absent from keyboards, so additional domain knowledge is required for people to figure out how to produce them. So instead, people developed all sorts of formats that are subject to delimiter collision.
Also, ASCII is recognized as a widely-deployed standard now, but this wasn't always the case. Computers used dozens of different codepages to represent characters with bytes, and while 26 English letters, 0-9, and some punctuation was always present, control characters seldom had equivalents in a different codepage, so interchange was a problem, because in most machines' native codepages, these delimiters were absent.
ASCII actually abbreviates 'American Standard Code for Information Interchange', but it largely came to be used for printable characters only -- "plain text", and not as a format for structured data.
Although by that point, the ship on C0 delimiters had largely sailed, to compound the chicken-and-egg problem, some codepages that were developed after ASCII often discarded the notion of control characters entirely, and redefined their byte sequences as additional printable characters. Windows-1252 was a notable offender [1].
[1] https://en.wikipedia.org/wiki/Windows-1252 https://en.wikipedia.org/wiki/Windows-1252
- derekp7 9y agoAs far as them showing up, at least in VIM they show as a caret followed by 64+code (so ^^ and ^_ for RS and US), and can be entered by ctrl v ctrl ^ and ctrl v ctrl _. Kind of awkward to enter though. Of course, like you said, this is domain-specific knowledge. I've converted much of my workflow to use these (I have a script that un-escapes data from mysql, and replaces tabs/newlines with RS and US). Most filter commands can have an arbitrary field delimiter specified, but not all of them can have something other than newline or null for a line terminator. For these cases I've had to maintain local copies of the commands (sed, sort, join).