4 ms·
Not to sound too factious but the decision to redesign the string API was a big reason why Python 3 has been a little on the slow cooked side. In for a penny an
by lucozade 11y ago
Not to sound too factious but the decision to redesign the string API was a big reason why Python 3 has been a little on the slow cooked side. In for a penny and all that.
If I recall correctly the modifier on it's own isn't correct. There is a base that should go before it to produce a valid character. I don't recall the base unfortunately and I don't have the spec to hand.
- klodolph 11y agoWhat do you mean by "correct"? The Unicode standard talks about well-formed UTF-8/16/32, but in general, any sequence of code points is permissible. By derivation, any sequence of code points can appear in JSON strings, and any sequence of code points excluding control characters can appear in XML. I personally have read the Unicode spec, and used it to implement some of the algorithms presented. This includes the grapheme cluster breaking algorithm... the case of combining characters at the beginning of a line is, in fact, covered by the spec, and cause a break both before and afterwards. In general, there are other cases where grapheme cluster breaks are not preserved by string concatenation, and you often want a locale-specific version anyway.
- lucozade 11y ago> What do you mean by "correct"? I mean this from the standard: "Nonspacing combining marks used by the Unicode Standard may be exhibited in apparent isolation by applying them to U+00A0 no-break space. " You can have valid Unicode that doesn't do this. But it's not intended to represent the character as some kind of textual thing. It's intended so that you can build up Unicode strings sensibly. Same idea as allowing mis-matched surrogates.