5 ms·

They're all similar to English. Languages have their syntactic ambiguities in different places, but they're all fundamentally similar. In every case the hard pa

They're all similar to English. Languages have their syntactic ambiguities in different places, but they're all fundamentally similar. In every case the hard part is in understanding the semantics expressed, and the characters used to express it are a side-issue.

TazeTSchnitzel 7y ago
This is assuming the problems are similar between languages. But are they? An NLP system tested only on English might be useless on CJK languages, because they do not use spaces, so the system cannot rely on the almost-free segmentation you get from English. Another example is that if you try a heavily-inflected language, you suddenly have vastly more forms of the same word than in English, and your system needs to be robust to that in a way that's unnecessary for English.
- deleted 7y ago
  [deleted]
- Veedrac 7y ago
  An NLP model that can't even (implicitly) segment CJK languages is laughable. It's like saying ‘sure, Magnus Carlsen is really good at chess, but in draughts you can take multiple moves in a row.’ If you can handle the ambiguities in natural casual English, you can handle a little inflection.
_emacsomancer_ 7y ago
They're similar at the atomic level, so to speak, but not very similar at the levels relevant for translation. The characters used to express languages are certainly a side-issue, but I don't understand why you would bring that into a discussion of syntactic structure. For compositional semantics, understanding the syntactic structures involved is crucial. Otherwise you end up with gibberish (i.e. what I find so often with Google Translate).
- Veedrac 7y ago
  My claim isn't that it's easy to solve NLP; of course a resource-constrained system like Google Translate that inderects many language pairs through English is going to have glaring issues. My claim is that the challenges generalize between languages. If you can handle English, you can handle any other language, and while the easy parts might at least look different at the surface, the hard parts are all the same.

TazeTSchnitzel 7y ago

This is assuming the problems are similar between languages. But are they? An NLP system tested only on English might be useless on CJK languages, because they do not use spaces, so the system cannot rely on the almost-free segmentation you get from English. Another example is that if you try a heavily-inflected language, you suddenly have vastly more forms of the same word than in English, and your system needs to be robust to that in a way that's unnecessary for English.

deleted 7y ago
[deleted]
Veedrac 7y ago
An NLP model that can't even (implicitly) segment CJK languages is laughable. It's like saying ‘sure, Magnus Carlsen is really good at chess, but in draughts you can take multiple moves in a row.’ If you can handle the ambiguities in natural casual English, you can handle a little inflection.

_emacsomancer_ 7y ago

They're similar at the atomic level, so to speak, but not very similar at the levels relevant for translation. The characters used to express languages are certainly a side-issue, but I don't understand why you would bring that into a discussion of syntactic structure. For compositional semantics, understanding the syntactic structures involved is crucial. Otherwise you end up with gibberish (i.e. what I find so often with Google Translate).

Veedrac 7y ago
My claim isn't that it's easy to solve NLP; of course a resource-constrained system like Google Translate that inderects many language pairs through English is going to have glaring issues. My claim is that the challenges generalize between languages. If you can handle English, you can handle any other language, and while the easy parts might at least look different at the surface, the hard parts are all the same.