Just on the face of it, as someone who has implemented a handful of tokenizers over time and can read Hangul, the Korean example there it four tokens, not twelv

Just on the face of it, as someone who has implemented a handful of tokenizers over time and can read Hangul, the Korean example there it four tokens, not twelve. You can break that down more with semantic parsing, but tokenization is not semantic parsing.