2 ms·

I also imagine that distinguishing between many of these languages is not a trivial task for a computer to handle.

• TillE 12y ago
  You'd get collisions if you're trying to determine the language of a single word, obviously. But a sentence or two should be no problem. You're just matching words in a dictionary. Different languages are mostly quite distinct. It would be a more interesting challenge to recognize and understand, say, German dialects.
  • a3_nm 12y ago
    In fact you can already get very good results, even without a dictionary, just with a distribution of the trigrams in your input languages (which is immediate to compute from a corpus). Just compute the distribution of the sample and return the language that has the closest one.
    • tripzilch 12y ago
      Even a single letter frequency histogram is already a very powerful discriminator to detect language. Although I suppose scoring maximum likelihood on bi- or trigrams works better for very small (sub-sentence) bits of text.