3 ms·
Wouldn't converting to nfkd/c first solve this issue too? My understanding of those forms was that they're made exactly for this case.
by iforgotpassword 6y ago
Wouldn't converting to nfkd/c first solve this issue too? My understanding of those forms was that they're made exactly for this case.
- jwilk 6y agoNo, these are ASCII strings, so they are already normalized.
- iforgotpassword 6y agoOh, I haven't used python much, but I thought it's all Unicode? If this were ascii it would work out of the box since there is no dotless lowercase i in ascii.
- estebank 6y agoThere are no code point for TURKISH LOWERCASE DOTTED I not for TURKISH UPPERCASE DOTLESS I, which means that the text doesn't carry enough information for roundtrip preservation. I believe this has proven to be a mistake but I'm not an expert. I don't know why it wasn't done.
- brewmarche 6y agoCase mapping and case folding are independent of normalization (in practice and it is the case here, see the end of SpecialCasing.txt) There is a good Unicode FAQ on the topic: < http://unicode.org/faq/casemap_charprop.html http://unicode.org/faq/casemap_charprop.html > E: to elaborate, I'm not sure whether the independence of case handling and normalization is guaranteed anywhere, and if we for example were to change the uppercase of ſ to something else than S then its compatibility forms' (s) case handling would differ. In practice the SpecialCasing.txt is designed to "make it work" (e.g. ſ uppercases to S).