2 ms·
This is assuming the problems are similar between languages. But are they? An NLP system tested only on English might be useless on CJK languages, because they
by TazeTSchnitzel 7y ago
This is assuming the problems are similar between languages. But are they? An NLP system tested only on English might be useless on CJK languages, because they do not use spaces, so the system cannot rely on the almost-free segmentation you get from English. Another example is that if you try a heavily-inflected language, you suddenly have vastly more forms of the same word than in English, and your system needs to be robust to that in a way that's unnecessary for English.
- deleted 7y ago[deleted]
- Veedrac 7y agoAn NLP model that can't even (implicitly) segment CJK languages is laughable. It's like saying ‘sure, Magnus Carlsen is really good at chess, but in draughts you can take multiple moves in a row.’ If you can handle the ambiguities in natural casual English, you can handle a little inflection.