5 ms·

We'll start to see the re-emergence of tools from old-school NLP, but now augmented with the powerful statistical tools and data-oriented automation of new-scho

We'll start to see the re-emergence of tools from old-school NLP, but now augmented with the powerful statistical tools and data-oriented automation of new-school NLP. IBM's Watson already does this to some extent. This is not a new trend. As early as 1997, Steven Abney augmented [1] attribute-value grammars with discriminative modelling (maximum entropy models) in this case to form 'stochastic attribute-value grammars'. There is a lot of work on efficiently extracting the best parse from packed forests, etc. Most systems that rely on unification grammars (e.g. HPSG grammars) already use stochastic models. In the early to mid 2000s when the modelling of association strengths using structured or unstructured text became popular, old-school parsers have been adopting such techniques to learn selectional preferences that cannot be learnt from the usually small hand-annotated treebanks. E.g. in languages that normally have SVO (subject-verb-object) for main clauses but also permit OVS order, parsers trained on small hand-annotated treebanks would often be set on the wrong path when the direct object is fronted (analyzing the direct object as subject). Techniques from association strength modelling were used to learn selectional preferences such as 'bread is usually the subject of eat' from automatically annotated text [2]. In recent years, learning word vector representations using neural networks has become popular. Again, not surprisingly, people have been integrating vectors as features in the disambiguation components of old-school NLP parsers. In some cases with great success. tl;dr, the flow of ideas and tools from new-school NLP to old-school NLP has been going on ever since the statistical NLP revolution started. [1] http://ucrel.lancs.ac.uk/acl/J/J97/J97-4005.pdf http://ucrel.lancs.ac.uk/acl/J/J97/J97-4005.pdf [2] http://www.let.rug.nl/vannoord/papers/iwptbook.pdf http://www.let.rug.nl/vannoord/papers/iwptbook.pdf

psygnisfive 12y ago
Yep, it's certainly true that these techniques never went away fully, but they've been eclipsed by the simpler techniques. Academic work, especially under the guise of "computational linguistics" as opposed to "NLP", works a lot on this. But the public face of NLP has been pure new-school stuff for a while now. I was more trying to just talk about why the richly structured old school fell from grace, not claim that it's dead and buried.
- danieldk 12y ago
  The rest of the analysis seemed very much to the point. It was an interesting read, thanks!
  - psygnisfive 12y ago
    I try. :) I've written some related stuff on the blog there, if you want more (most of it I've linked here on HN?) I'm gonna try to write more often, so hopefully I'll keep giving you interesting reads! :)
SixSigma 12y ago
The problem is slippery though, people always reuse words to express different ideas : "We can't buy any bread because we haven't got any bread." And it's not just English. In Chinese one is taught that "ni hao ma?" is the greeting equivalent to "hello, how are you" but try it on a Chinese person and it amuses them. My Chinese friend at Uni says that Chinese people use "Ni chi ma?" which is literally "have you eaten?" (although we both appreciate that is a bit of a generalisation for 1bn people).
- cynicalkane 12y ago
  The Taiwanese say "ni hao ma". That's probably where pedagogy got it from.
  - SixSigma 12y ago
    I imagine plenty of Chinese say it too, really, just like we are taught in my Chinese class at Uni. The point being though, that even if you are taught "Hello, how are you" in an English class then you could well be stumped when you come to my region and we say "How's tricks?"
    - danieldk 12y ago
      The referenced papers are about syntactic parsing, not semantics.
- danieldk 12y ago
  > The problem is slippery though, people always reuse words to express different ideas : > "We can't buy any bread because we haven't got any bread." As long as we are talking about syntactic parsing, this is not a problem as long as the attachment is the same. In both cases 'bread' is the direct object of the main verb. Of course, there are cases where a particular word can be used both as a direct object and a subject of a particular verb. E.g.: The man ate the pig. The pig ate the apple. Of course, what such systems are learning are not rules, but probability distributions that combine information about the distributions of word orders, association strengths between heads and dependent with a particular dependency relation, configurations of dependent pairs, etc.

psygnisfive 12y ago

Yep, it's certainly true that these techniques never went away fully, but they've been eclipsed by the simpler techniques. Academic work, especially under the guise of "computational linguistics" as opposed to "NLP", works a lot on this. But the public face of NLP has been pure new-school stuff for a while now. I was more trying to just talk about why the richly structured old school fell from grace, not claim that it's dead and buried.

danieldk 12y ago
The rest of the analysis seemed very much to the point. It was an interesting read, thanks!
- psygnisfive 12y ago
  I try. :) I've written some related stuff on the blog there, if you want more (most of it I've linked here on HN?) I'm gonna try to write more often, so hopefully I'll keep giving you interesting reads! :)

SixSigma 12y ago

The problem is slippery though, people always reuse words to express different ideas : "We can't buy any bread because we haven't got any bread." And it's not just English. In Chinese one is taught that "ni hao ma?" is the greeting equivalent to "hello, how are you" but try it on a Chinese person and it amuses them. My Chinese friend at Uni says that Chinese people use "Ni chi ma?" which is literally "have you eaten?" (although we both appreciate that is a bit of a generalisation for 1bn people).

cynicalkane 12y ago
The Taiwanese say "ni hao ma". That's probably where pedagogy got it from.
- SixSigma 12y ago
  I imagine plenty of Chinese say it too, really, just like we are taught in my Chinese class at Uni. The point being though, that even if you are taught "Hello, how are you" in an English class then you could well be stumped when you come to my region and we say "How's tricks?"
  - danieldk 12y ago
    The referenced papers are about syntactic parsing, not semantics.
danieldk 12y ago
> The problem is slippery though, people always reuse words to express different ideas : > "We can't buy any bread because we haven't got any bread." As long as we are talking about syntactic parsing, this is not a problem as long as the attachment is the same. In both cases 'bread' is the direct object of the main verb. Of course, there are cases where a particular word can be used both as a direct object and a subject of a particular verb. E.g.: The man ate the pig. The pig ate the apple. Of course, what such systems are learning are not rules, but probability distributions that combine information about the distributions of word orders, association strengths between heads and dependent with a particular dependency relation, configurations of dependent pairs, etc.