3 ms·
Very nice. It would be considerably enhanced by some examples of how the algorithm performs on real data. (I wonder how it does if you feed it not words from a
by gjm11 16y ago
Very nice. It would be considerably enhanced by some examples of how the algorithm performs on real data.
(I wonder how it does if you feed it not words from a natural language, but tokens from a programming language. Can this sort of technique be adapted to infer the whole grammar? I guess that would be more difficult -- you're trying to learn a much more complicated sort of structure.)
- bravura 16y agoThis is an unsupervised part-of-speech tagger, not an unsupervised parser. Grammar induction is not possible in Aria's model, assuming we are talking about PCFG or some other hierarchical grammar.
- alextp 16y agoIf you feed it words and appropriate features from a programming language you should get something close to a tokenizer. Laura dietz ( http://www.mpi-inf.mpg.de/~dietz/index.html#publications http://www.mpi-inf.mpg.de/~dietz/index.html#publications ) has some work that applies similar techniques to programming languages to find bugs.