4 ms·

Testing a statistical language identifier with texts this short is absurd. If you type in four or five words from https://en.wikipedia.org/wiki/List_of_English

• andreasvc 12y ago
  It is not absurd. Generally, if humans can do it, it is a reasonable task for NLP to attempt. Yes you can present edge cases where there is no definite answer, like the one you cite, but this doesn't mean that the task in general is impossible or useless.
  • wooorm 12y ago
    I agree the task is neither impossible nor useless. There’s work to do. Short passages should be supported. I do however think franc does a good job, and adds support for some languages which before today have never (I think) been supported. Franc, certainly, “attempt”s to fix language detection, which I would argue is an AI-complete problem.