3 ms·
Linguistic identification can in principle be a bit smarter than that. Aside from the obvious "if it has kana it is Japanese" metric, there are also differences
by adastra22 3y ago
Linguistic identification can in principle be a bit smarter than that. Aside from the obvious "if it has kana it is Japanese" metric, there are also differences in code point choice which differentiate Simplified, Traditional, and Japanese. Fully accurate identification is certainly expensive, but quick early-out heuristics could be developed.
- rsaxvc 3y agoThis reminds me of guessing codepoint likelihood based on character distribution.
- eviks 3y agoindeed it could be better, though do you know of any apps that actually try to be smarter?
- adastra22 3y agoIn principle this is a job for Unicode font rendering libraries and/or standard OS text widgets, not app developers.