4 ms·

Swift's String implementation is just porn to me. They had a few misguided attempts in Swift 1 through 3, but their final design is truly marvelous and points p

Swift's String implementation is just porn to me. They had a few misguided attempts in Swift 1 through 3, but their final design is truly marvelous and points programmers who have no idea about encodings towards the right solutions, like by simply counting grapheme clusters correctly, avoiding cloning thanks to views and not allowing for direct string[subscript] access without deliberately stepping down into utf8 or utf16 codepoint representations. The fact that Swift is aiming for ABI stability, something neither C++ nor Rust have because we all rely on C for FFI, is very interesting.

eridius 8y ago
Unfortunately, since Swift's String implementation operates on grapheme clusters by default, a lot of parsing code people write is actually subtly broken in the presence of combining characters. As a trivial example, let's say the input is a comma-delimited string (say, a line of CSV without quotes). The obvious way to split this is let fields = line.split(separator: ",") But given the input "foo,\u{301}bar" (which looks like foo,́bar), this won't split correctly and you'll end up with a single field that contains a comma. The correct way to split this is let fields = line.unicodeScalars.split(separator: ",").map(Substring.init) This will get you the correct 2 fields (at the cost of an intermediate array, as there is no lazy split).
- Twisell 8y ago
  Have you opened a ticket about that? This is more due to the split implementation than to the format itself. Room for improvement.
  - deleted 8y ago
    [deleted]
  - eridius 8y ago
    This is not an issue with the split implementation. Split is behaving correctly (according to the defined semantics of String). You'd get the exact same issue if you said str.index(of: ",") instead; in both cases, the "," is a Character, not a UnicodeScalar, and "," != ",\u{301}".
- masklinn 8y ago
  The original split looks correct to me. If you're assuming CSV is a textual format, a comma with a combining acute accent is not the same thing as a comma: if I'm asking for a split on "e", I don't want a split on "é" whether in its precomposed form or not, and I especially do not want lone combining characters floating around as a result. If you're assuming CSV is a binary format, you should split on code units before the textual decoding.
  - eridius 8y ago
    CSV is a machine-readable format. I've never heard of a machine-readable format that treats combining characters as significant when parsing syntax. Certainly the CSV format doesn't. Trivial proof: If it did, then it would be literally impossible to represent a field starting with a combining character. Also, the set of combining characters has changed over time. Machine-readable formats in general do not change as Unicode does. A CSV that parses today should not fail to parse next year because a field starts with a codepoint that today is unused and next year has been assigned to a combining character.
- dbaupp 8y ago
  "Parsing" CSV using split (whether on graphemes or on scalars) isn't correct at all, though, due to quoting.
  - eridius 8y ago
    This is why I said "without quotes". The same issue happens if you write a parsing routine that handles quotes too of course, it was just a lot simpler to demonstrate it with a single split() line than a whole parsing loop.
kjeetgill 8y ago
Views are ... Hit and miss. I think java has flip-floped on that one. The issue is cases where you parse a config to extract a single property name and that keeps the whole thing around.
- dwaite 8y ago
  String slices and views in swift have a distinct type from string. Implicit property types can bring this problem back, but it is at least harder to do by accident than in Java.
- masklinn 8y ago
  The problem of Java is that its views were implicit: a given `String` could be either the owner of its data entire, or a reference to a slice in an other string. This lead to cheap sub-stringing, but significant memory leaks as you'd carry around a short substring and it would carry along the entire string it originated from (possibly gigabytes of it). In Swift, string views are a different type from strings. For the most part you can perform similar operations on them but they are not the same type, so a `Substring` clearly tells you that it's linked to arbitrary-size baggage, while a `String` tells you it owns its data. You can operate generically over both using string protocols, or you can specifically ask for one or the other.
woolvalley 8y ago
And as a result, string performance is relatively horrible in swift when you don't need all of that, and you have to drop down to something approaching a c string to avoid it.

eridius 8y ago

Unfortunately, since Swift's String implementation operates on grapheme clusters by default, a lot of parsing code people write is actually subtly broken in the presence of combining characters. As a trivial example, let's say the input is a comma-delimited string (say, a line of CSV without quotes). The obvious way to split this is let fields = line.split(separator: ",") But given the input "foo,\u{301}bar" (which looks like foo,́bar), this won't split correctly and you'll end up with a single field that contains a comma. The correct way to split this is let fields = line.unicodeScalars.split(separator: ",").map(Substring.init) This will get you the correct 2 fields (at the cost of an intermediate array, as there is no lazy split).

Twisell 8y ago
Have you opened a ticket about that? This is more due to the split implementation than to the format itself. Room for improvement.
- deleted 8y ago
  [deleted]
- eridius 8y ago
  This is not an issue with the split implementation. Split is behaving correctly (according to the defined semantics of String). You'd get the exact same issue if you said str.index(of: ",") instead; in both cases, the "," is a Character, not a UnicodeScalar, and "," != ",\u{301}".
masklinn 8y ago
The original split looks correct to me. If you're assuming CSV is a textual format, a comma with a combining acute accent is not the same thing as a comma: if I'm asking for a split on "e", I don't want a split on "é" whether in its precomposed form or not, and I especially do not want lone combining characters floating around as a result. If you're assuming CSV is a binary format, you should split on code units before the textual decoding.
- eridius 8y ago
  CSV is a machine-readable format. I've never heard of a machine-readable format that treats combining characters as significant when parsing syntax. Certainly the CSV format doesn't. Trivial proof: If it did, then it would be literally impossible to represent a field starting with a combining character. Also, the set of combining characters has changed over time. Machine-readable formats in general do not change as Unicode does. A CSV that parses today should not fail to parse next year because a field starts with a codepoint that today is unused and next year has been assigned to a combining character.
dbaupp 8y ago
"Parsing" CSV using split (whether on graphemes or on scalars) isn't correct at all, though, due to quoting.
- eridius 8y ago
  This is why I said "without quotes". The same issue happens if you write a parsing routine that handles quotes too of course, it was just a lot simpler to demonstrate it with a single split() line than a whole parsing loop.

kjeetgill 8y ago

Views are ... Hit and miss. I think java has flip-floped on that one. The issue is cases where you parse a config to extract a single property name and that keeps the whole thing around.

dwaite 8y ago
String slices and views in swift have a distinct type from string. Implicit property types can bring this problem back, but it is at least harder to do by accident than in Java.
masklinn 8y ago
The problem of Java is that its views were implicit: a given `String` could be either the owner of its data entire, or a reference to a slice in an other string. This lead to cheap sub-stringing, but significant memory leaks as you'd carry around a short substring and it would carry along the entire string it originated from (possibly gigabytes of it). In Swift, string views are a different type from strings. For the most part you can perform similar operations on them but they are not the same type, so a `Substring` clearly tells you that it's linked to arbitrary-size baggage, while a `String` tells you it owns its data. You can operate generically over both using string protocols, or you can specifically ask for one or the other.

woolvalley 8y ago

And as a result, string performance is relatively horrible in swift when you don't need all of that, and you have to drop down to something approaching a c string to avoid it.