6 ms·

ASCII has special delimiters 0x1E Record Separator and 0x1F Unit Separator to avoid conflicting with values, but they have never gained widespread adoption.

CRConrad 5y ago
> ASCII has special delimiters 0x1E Record Separator and 0x1F Unit Separator And since ASCII has them, Unicode has them too.
nicoburns 5y ago
I worked on a project that used these in a proprietary file format. It was actually quite nice. There was no escaping required because they simply stripped those characters out of the actual content.
hermitcrab 5y ago
While these have clear advantages over comma and CR for parsing (no more escaping!), they aren't at all convenient for manual editing.
- mavhc 5y ago
  If they were popular they'd be supported in your editor though.
  - hermitcrab 5y ago
    Maybe. It is a bit of a chicken-and-egg problem though. They aren't likely to become popular until editors support them. Also, the fact that they have been around for years and never taken off implies that they probably never will.
    - sli 5y ago
      More generally than editor support, how is the average person expected to type them? I could easily add them to my keyboard, but if "editor support" turned out to mean "Edit > Insert > Record Separator Character" and I was now bound only to editors that support this, absolutely nobody is going to bother. They specifically need to be trivially typable in exactly the way commas and newlines are. Realistically I don't think anyone (at least on HN) would presume that editor support would be so weirdly obtuse, but it's not exactly well-defined here, either. I'm mostly just thinking out loud, here, not really critiquing anyone.
      - kevin_thibedeau 5y ago
        > how is the average person expected to type them? [modifier]-[C,D,E,F]
        xkgt 5y ago
        which modifier would that be, Alt/Ctrl or Cmd? Also there is a chance that this combo will not work on some platforms, some may already be in use as a hotkey.
        epmos 5y ago
        Actually, there is already a standard way for someone to type these with the control key. To be specific, they are: FS: ^\ GS: ^] RS: ^^ US: ^_ Since these delimiters are not much used, on many terminals ^\ is mapped to send SIGQUIT. Use 'stty quit undef' and you can then type this at the keyboard. This works on Linux and OS X, and likely most other systems. Note that with most keyboards you are also using shift for RS and US. For emacs, prefix with C-q so that it inserts the literal. Other posters have noted that they have ASCII separators recently, I have as recently as 2018 internally at a FAANG. So they are not completely unknown in the wild.
      - mavhc 5y ago
        Commas and Newlines are only trivial if your data contains no commas or newlines. Who's writing CSV files by hand though? I'd expect some kind of CSV mode in my text editor where Tab and Newlines are turned into keys to insert those characters
a1369209993 5y ago
> but they have never gained widespread adoption. Yes, because that just kicks the can down the road: how do you represent the one-byte string "\x1E"?
- BenjiWiebe 5y ago
  At least very few humans type that into their spreadsheet cell.

CRConrad 5y ago

> ASCII has special delimiters 0x1E Record Separator and 0x1F Unit Separator And since ASCII has them, Unicode has them too.

nicoburns 5y ago

I worked on a project that used these in a proprietary file format. It was actually quite nice. There was no escaping required because they simply stripped those characters out of the actual content.

hermitcrab 5y ago

While these have clear advantages over comma and CR for parsing (no more escaping!), they aren't at all convenient for manual editing.

mavhc 5y ago
If they were popular they'd be supported in your editor though.
- hermitcrab 5y ago
  Maybe. It is a bit of a chicken-and-egg problem though. They aren't likely to become popular until editors support them. Also, the fact that they have been around for years and never taken off implies that they probably never will.
  - sli 5y ago
    More generally than editor support, how is the average person expected to type them? I could easily add them to my keyboard, but if "editor support" turned out to mean "Edit > Insert > Record Separator Character" and I was now bound only to editors that support this, absolutely nobody is going to bother. They specifically need to be trivially typable in exactly the way commas and newlines are. Realistically I don't think anyone (at least on HN) would presume that editor support would be so weirdly obtuse, but it's not exactly well-defined here, either. I'm mostly just thinking out loud, here, not really critiquing anyone.
    - kevin_thibedeau 5y ago
      > how is the average person expected to type them? [modifier]-[C,D,E,F]
      - xkgt 5y ago
        which modifier would that be, Alt/Ctrl or Cmd? Also there is a chance that this combo will not work on some platforms, some may already be in use as a hotkey.
      - epmos 5y ago
        Actually, there is already a standard way for someone to type these with the control key. To be specific, they are: FS: ^\ GS: ^] RS: ^^ US: ^_ Since these delimiters are not much used, on many terminals ^\ is mapped to send SIGQUIT. Use 'stty quit undef' and you can then type this at the keyboard. This works on Linux and OS X, and likely most other systems. Note that with most keyboards you are also using shift for RS and US. For emacs, prefix with C-q so that it inserts the literal. Other posters have noted that they have ASCII separators recently, I have as recently as 2018 internally at a FAANG. So they are not completely unknown in the wild.
    - mavhc 5y ago
      Commas and Newlines are only trivial if your data contains no commas or newlines. Who's writing CSV files by hand though? I'd expect some kind of CSV mode in my text editor where Tab and Newlines are turned into keys to insert those characters

a1369209993 5y ago

> but they have never gained widespread adoption. Yes, because that just kicks the can down the road: how do you represent the one-byte string "\x1E"?

BenjiWiebe 5y ago
At least very few humans type that into their spreadsheet cell.