3 ms·

Two links away is the answer: https://github.com/SixArm/usv/tree/main/doc/faq#why-use-control-picture-characters-rather-than-the-control-characters-themselves h

Two links away is the answer: https://github.com/SixArm/usv/tree/main/doc/faq#why-use-control-picture-characters-rather-than-the-control-characters-themselves https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...

divbzero 3y ago
The answer makes sense to me, but I wish we could fix editors to properly handle the ASCII separators (1C, 1D, 1E, 1F) instead of resorting to Unicode control picture characters (241C, 241D, 241E, 241F). Maybe if editors are fixed up we could adopt ASCII Separated Values (ASV) as the new standard.
- throwanem 3y ago
  Emacs has handled literal ASCII control characters correctly I believe since around the time I was born - probably somewhat earlier, if we count back further than GNU. Unicode works fine there too, so it makes no nevermind to me which flavor people use. I just think it's funny how "everything old is new again".
- jph 3y ago
  Yes you're right. That's a long term goal.
marwis 3y ago
Why not combine zero width character with visible character, i.e. use 2 characters for separators? ,<FS> for fields \n<RS> for records This removes ambiguity in parsing and remains user readable. It's also relatively easy to auto-fix files edited by users in normal editors. It also mostly removes need for escaping. It's also smaller or same size as unicode multibyte characters (haven't checked).
- Renaud 3y ago
  Wouldn't this easily break your file in subtle ways when someone tries to edit it in their editor and the zero width character is not visible? How could you make the difference with a standard CSV file if it looks like a standard CSV file? They explain why they don't use control characters. Editors are not consistent in how they show control/zero-length characters: https://github.com/SixArm/usv/tree/main/doc/faq#why-use-control-picture-characters-rather-than-the-control-characters-themselves https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...

divbzero 3y ago

The answer makes sense to me, but I wish we could fix editors to properly handle the ASCII separators (1C, 1D, 1E, 1F) instead of resorting to Unicode control picture characters (241C, 241D, 241E, 241F). Maybe if editors are fixed up we could adopt ASCII Separated Values (ASV) as the new standard.

throwanem 3y ago
Emacs has handled literal ASCII control characters correctly I believe since around the time I was born - probably somewhat earlier, if we count back further than GNU. Unicode works fine there too, so it makes no nevermind to me which flavor people use. I just think it's funny how "everything old is new again".
jph 3y ago
Yes you're right. That's a long term goal.

marwis 3y ago

Why not combine zero width character with visible character, i.e. use 2 characters for separators? ,<FS> for fields \n<RS> for records This removes ambiguity in parsing and remains user readable. It's also relatively easy to auto-fix files edited by users in normal editors. It also mostly removes need for escaping. It's also smaller or same size as unicode multibyte characters (haven't checked).

Renaud 3y ago
Wouldn't this easily break your file in subtle ways when someone tries to edit it in their editor and the zero width character is not visible? How could you make the difference with a standard CSV file if it looks like a standard CSV file? They explain why they don't use control characters. Editors are not consistent in how they show control/zero-length characters: https://github.com/SixArm/usv/tree/main/doc/faq#why-use-control-picture-characters-rather-than-the-control-characters-themselves https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...