2 ms·
Normalization covers more than just combining characters. CJK full width digits and punctuation can be problematic when you want the canonical forms for pattern
by kevin_thibedeau 8y ago
Normalization covers more than just combining characters. CJK full width digits and punctuation can be problematic when you want the canonical forms for pattern matching:
s = '1 2'
print(unicodedata.normalize('NFD', s))
print(unicodedata.normalize('NFC', s))
print(unicodedata.normalize('NFKC', s))
1 2
1 2
1 2
- lelf 8y agoIt's called collation > $*COLLATION.set(:!secondary:!tertiary:!quaternary) collation-level => 1, Country => International, Language => None, primary => 1, secondary => 0, tertiary => 0, quaternary => 0 > '12' coll '12' Same