3 ms·

It's much less likely that content is going to include ASCII 28 to 31 than that it will include a comma or a tab. But you're right, certainly not impossible. I

It's much less likely that content is going to include ASCII 28 to 31 than that it will include a comma or a tab. But you're right, certainly not impossible. I think the human-editability you mention may be a major reason why comma-separated has been such a practical winner? Since ASCII 28 to 31 are invisible or indistinguishable in most editors, I suspect a format relying on them would be subject to routine corruption. (I have some experience working with a "legacy" "binary" format which DOES use those ASCII separator values... https://en.wikipedia.org/wiki/MARC_standards https://en.wikipedia.org/wiki/MARC_standards)

Nullabillity 5y ago
> It's much less likely that content is going to include ASCII 28 to 31 than that it will include a comma or a tab. But you're right, certainly not impossible. Yeah, that's the same problem as UTF-8's almost-compatibility with ASCII. Things work, until they break spectacularly. Ultimately, you either have to solve the issue properly, or keep it visible enough that people are forced to handle it.
slver 5y ago
In machine encoding formats, "much less likely" is the same as "very likely". You're either OK for something to happen or you categorically make it impossible to happen. Invisible characters also mean you can no longer call a data format "human readable". The special ASCII characters you're referring to are mostly a historical oddity today.
- jrochkind1 5y ago
  How much have you worked with actually existing CSV data? If it was as easy as "making it categorically impossible to happen", working with CSV wouldn't be such a mess! You can say whatever you want in whatever standard you want, data bugs still happen. Any CSV format description anywhere, will of course you need to use various internal escaping mechanisms for a literal comma, or forbid them entirely. Does that "categorically make it impossible to happen"? Obviously not, words in a standard can't make something categorically impossible. How likely a protocol/standard violation is to happen of course matters to actual practical pain. Yes, of course the ascii separator values are a historical path not taken. This thread, not started by me, was musing about why, and what-if, by people who, believe it or not, have plenty of experience dealing with data, and bad data.
  - slver 5y ago
    You’ve misinterpreted my comment. And arguing about unrelated things.

Nullabillity 5y ago

> It's much less likely that content is going to include ASCII 28 to 31 than that it will include a comma or a tab. But you're right, certainly not impossible. Yeah, that's the same problem as UTF-8's almost-compatibility with ASCII. Things work, until they break spectacularly. Ultimately, you either have to solve the issue properly, or keep it visible enough that people are forced to handle it.

slver 5y ago

In machine encoding formats, "much less likely" is the same as "very likely". You're either OK for something to happen or you categorically make it impossible to happen. Invisible characters also mean you can no longer call a data format "human readable". The special ASCII characters you're referring to are mostly a historical oddity today.

jrochkind1 5y ago
How much have you worked with actually existing CSV data? If it was as easy as "making it categorically impossible to happen", working with CSV wouldn't be such a mess! You can say whatever you want in whatever standard you want, data bugs still happen. Any CSV format description anywhere, will of course you need to use various internal escaping mechanisms for a literal comma, or forbid them entirely. Does that "categorically make it impossible to happen"? Obviously not, words in a standard can't make something categorically impossible. How likely a protocol/standard violation is to happen of course matters to actual practical pain. Yes, of course the ascii separator values are a historical path not taken. This thread, not started by me, was musing about why, and what-if, by people who, believe it or not, have plenty of experience dealing with data, and bad data.
- slver 5y ago
  You’ve misinterpreted my comment. And arguing about unrelated things.