4 ms·

Who implements transliteration rules? I assume operating systems? Or text renderers?

• kccqzy 3mo ago
  The ICU library. It’s so ubiquitous that it’s somewhat difficult to avoid this library if you are doing anything advanced with human text.
  • ks2048 3mo ago
    And ICU uses data from CLDR, which is mentioned in the blog. Here, there are 380 xml files: https://github.com/unicode-org/cldr/tree/main/common/transforms https://github.com/unicode-org/cldr/tree/main/common/transfo... Yes, ICU is ubiquitous. But, some NLP projects use various other libraries, such as uroman (just for romanization - to Latin script).
  • poizan42 3mo ago
    Windows has its own library built-in: NLS. There have been a lot of back-and-forth about whether to use NLS or ICU in dotnet on Windows because it resulted in different behavior between Windows and other platforms. I think they ended up using ICU by default from .NET 5 on.