4 ms·

ASCII also includes control characters for delimiting text records. If people used these purposed-designed characters instead of comma or tab characters as deli

ASCII also includes control characters for delimiting text records. If people used these purposed-designed characters instead of comma or tab characters as delimiters, we could avoid many headaches quoting and escaping CSV data. https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_separators https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_... ASCII 28 for File Separator ASCII 29 for Group Separator ASCII 30 for Record Separator ASCII 31 for Unit Separator

tlb 7y ago
And replace them with a whole new set of headaches, like difficult typing and copy-pasting.
- retrocryptid 7y ago
  Typing should be easy. Just type ctrl-], ctrl-/, ctrl-[ or ctrl-^. In the bash shell, you may need to type the quote character (ctrl-v) first, depending on your try settings. Copy and paste worked fine for me in lxterminal an xterm.
  - saagarjha 7y ago
    Typing those in anywhere that’s not a shell is still a pain.
    - Crinus 7y ago
      Isn't that mainly because these weren't norm? For the vast majority of text editing software you do not type Ctrl+J for a new line after all.
firethief 7y ago
Quoting/escaping would still be necessary, but the need would be less obvious. It would be deferred from a common annoyance at development time to a common cause of nasty surprises in production.
OskarS 7y ago
This is a terrible idea! You're not avoiding anything, you would still have to escape these characters in CSV because it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file. Besides, it's arguably a GOOD thing that the delimiter in CSV (the comma) is such a common character, because it forces all parsers to properly support escaping and quoting. If it used some sort of unique character that was almost never used except in CSV, then 99.9% it would work without correct escaping, but the 0.01% when someone entered "you should use the character X to separate fields" as a column it would fail, and those cases are much more likely to slip through the cracks. Also: the whole point of CSV is to be human readable. There's no obvious way to render "Record separator" on screen, and the format would essentially become a binary format.
- jgalt212 7y ago
  > it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file. Our business has parsed and emitted significant volumes of data have never encountered one of these code points in the wild. My understanding of why not to use these code points is that they don't have printable glyphs and thus don't easily lend themselves to ad hoc data exploration tools.
- tinus_hn 7y ago
  Unfortunately the comma is the decimal separator in some locales but the thousands separator in others. As the US doesn’t really use commas in floating point numbers escaping these is often an afterthought causing issues that only surface abroad. Another reason why CSV is a terrible format.

tlb 7y ago

And replace them with a whole new set of headaches, like difficult typing and copy-pasting.

retrocryptid 7y ago
Typing should be easy. Just type ctrl-], ctrl-/, ctrl-[ or ctrl-^. In the bash shell, you may need to type the quote character (ctrl-v) first, depending on your try settings. Copy and paste worked fine for me in lxterminal an xterm.
- saagarjha 7y ago
  Typing those in anywhere that’s not a shell is still a pain.
  - Crinus 7y ago
    Isn't that mainly because these weren't norm? For the vast majority of text editing software you do not type Ctrl+J for a new line after all.

firethief 7y ago

Quoting/escaping would still be necessary, but the need would be less obvious. It would be deferred from a common annoyance at development time to a common cause of nasty surprises in production.

OskarS 7y ago

This is a terrible idea! You're not avoiding anything, you would still have to escape these characters in CSV because it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file. Besides, it's arguably a GOOD thing that the delimiter in CSV (the comma) is such a common character, because it forces all parsers to properly support escaping and quoting. If it used some sort of unique character that was almost never used except in CSV, then 99.9% it would work without correct escaping, but the 0.01% when someone entered "you should use the character X to separate fields" as a column it would fail, and those cases are much more likely to slip through the cracks. Also: the whole point of CSV is to be human readable. There's no obvious way to render "Record separator" on screen, and the format would essentially become a binary format.

jgalt212 7y ago
> it's entirely possible (in fact, almost certain) that someone will want to use the character itself in a CSV file. Our business has parsed and emitted significant volumes of data have never encountered one of these code points in the wild. My understanding of why not to use these code points is that they don't have printable glyphs and thus don't easily lend themselves to ad hoc data exploration tools.
tinus_hn 7y ago
Unfortunately the comma is the decimal separator in some locales but the thousands separator in others. As the US doesn’t really use commas in floating point numbers escaping these is often an afterthought causing issues that only surface abroad. Another reason why CSV is a terrible format.