3 ms·

Thank you! The ICU Boundary Analysis documentation says it uses a dictionary to split Chinese, Japanese, Thai or Khmer. https://unicode-org.github.io/icu/userg

Thank you! The ICU Boundary Analysis documentation says it uses a dictionary to split Chinese, Japanese, Thai or Khmer. https://unicode-org.github.io/icu/userguide/boundaryanalysis/ https://unicode-org.github.io/icu/userguide/boundaryanalysis... Is that the same as the macOS dictionary being parsed here? It seems like a pretty big file to grep every time!

krackers 5y ago
No, the ICU dictionaries are seen at: https://github.com/unicode-org/icu/tree/main/icu4c/source/data/brkitr/dictionaries https://github.com/unicode-org/icu/tree/main/icu4c/source/da... I assume at compile time it's converted to a more efficient query format