3 ms·

Yes you're right. For special character escaping like your blog post idea, have you seen Unicode Separated Values Extensions (USVX)? It's USV plus extras for ba

by jph 4y ago

Yes you're right. For special character escaping like your blog post idea, have you seen Unicode Separated Values Extensions (USVX)? It's USV plus extras for backslash escape, whitespace trim, etc. These use a smarter parser.

kazinator 4y ago
The language being embedded should provide these features, so USV is kept simple. If the data is HTML, then you can encode the special characters using ampersand notation. And that can be kept out of the USV spec entirely as an external convention. So that is to say, use the native escaping of whatever data format you are storing. A similar principle is used in embedding JS or JSON in HTML. A sequence like "</script" could occur in a JS or JSON string literal. For that reason there is a way to escape the backslash \/. Or else the slash can be encoded numerically. This is all provided by the embedded language; the embedding language (HTML) knows nothing about it. The user, or software, doing the embedding just has to know about this and put in the escapes if they occur in the to-be-embedded data. USV is nice the way it is; don't make a complicated skullduggery out of it with escape mechanisms. The whole point of choosing those characters is to avoid it. If you're gonna have escapes, then might as well stick with commmas for field separation, newlines for records.
- jph 4y ago
  I hear you. The feedback from this post and other testers is approximately evenly split. Some people like the simplicity, some people want/need extras especially whitespace trim, backspace escape, and final newline. What's your opinion of USV being simple (as you describe above) and USVX being a different standard that is USV + extras?
  - kazinator 4y ago
    USV is not simple if USVX exists. There is a potential for confusion. E.g. someone carelessly documents that some software accepts USV, when they mean USVX, or vice versa. You will forever have to be double-checking. Imagine if we had JSON and JSONX to contend with. White space trim means that if you don't expect trim, your significant white space is eaten, causing a bug. Most of the time, USV will be machine-generated. If the machine generates a field with leading or trailing space, then that's what the data is.

by jph 4y ago

kazinator 4y ago

The language being embedded should provide these features, so USV is kept simple. If the data is HTML, then you can encode the special characters using ampersand notation. And that can be kept out of the USV spec entirely as an external convention. So that is to say, use the native escaping of whatever data format you are storing. A similar principle is used in embedding JS or JSON in HTML. A sequence like "</script" could occur in a JS or JSON string literal. For that reason there is a way to escape the backslash \/. Or else the slash can be encoded numerically. This is all provided by the embedded language; the embedding language (HTML) knows nothing about it. The user, or software, doing the embedding just has to know about this and put in the escapes if they occur in the to-be-embedded data. USV is nice the way it is; don't make a complicated skullduggery out of it with escape mechanisms. The whole point of choosing those characters is to avoid it. If you're gonna have escapes, then might as well stick with commmas for field separation, newlines for records.

jph 4y ago
I hear you. The feedback from this post and other testers is approximately evenly split. Some people like the simplicity, some people want/need extras especially whitespace trim, backspace escape, and final newline. What's your opinion of USV being simple (as you describe above) and USVX being a different standard that is USV + extras?
- kazinator 4y ago
  USV is not simple if USVX exists. There is a potential for confusion. E.g. someone carelessly documents that some software accepts USV, when they mean USVX, or vice versa. You will forever have to be double-checking. Imagine if we had JSON and JSONX to contend with. White space trim means that if you don't expect trim, your significant white space is eaten, causing a bug. Most of the time, USV will be machine-generated. If the machine generates a field with leading or trailing space, then that's what the data is.