3 ms·
The original split looks correct to me. If you're assuming CSV is a textual format, a comma with a combining acute accent is not the same thing as a comma: if I
by masklinn 8y ago
The original split looks correct to me. If you're assuming CSV is a textual format, a comma with a combining acute accent is not the same thing as a comma: if I'm asking for a split on "e", I don't want a split on "é" whether in its precomposed form or not, and I especially do not want lone combining characters floating around as a result.
If you're assuming CSV is a binary format, you should split on code units before the textual decoding.
- eridius 8y agoCSV is a machine-readable format. I've never heard of a machine-readable format that treats combining characters as significant when parsing syntax. Certainly the CSV format doesn't. Trivial proof: If it did, then it would be literally impossible to represent a field starting with a combining character. Also, the set of combining characters has changed over time. Machine-readable formats in general do not change as Unicode does. A CSV that parses today should not fail to parse next year because a field starts with a codepoint that today is unused and next year has been assigned to a combining character.