4 ms·
Just a note that normalization is not the same as diacritic insensitivity. Normalization is the process by which strings that are semantically equivalent (by so
by bobbylarrybobby 5y ago
Just a note that normalization is not the same as diacritic insensitivity. Normalization is the process by which strings that are semantically equivalent (by some standard), yet have different underlying byte sequences, are transformed to have the same underlying byte sequence. For instance, replacing “e, combining acute accent” with “e with acute accent”.
- dhosek 5y agoYes, I should clarify that the process is to normalize to letter+combining accent and then strip out the accents, e.g., in Java: Normalizer.normalize(text, Normalizer.Form.NFD).replaceAll("\\p{M}", ""); (I hope I got this right—I adapted it from some code which actually strips out some other bits of text to turn, e.g., F. Janaček into FJanacek.)