3 ms·
>> Schuler’s team is working to build a Bayseian sequence model based on statistical analysis to discover a given language’s grammar. It is hypothesized this pa
by YeGoblynQueenne 9y ago
>> Schuler’s team is working to build a Bayseian sequence model based on
statistical analysis to discover a given language’s grammar. It is
hypothesized this parsing model can be trained to learn a language and make it
syntactically useful.
I'm not sure who is "hypothesising" this but the consensus in the field of
grammar induction [1] has long been that natural language learning from
examples is intractable [2].
I also see some really funny claims about "rapid grammar acquisition" from
just 20,000 sentences. For comparison, in English that'd be about 2.8 MB of
text (assuming sentences with an average of 25 words and words with an average
of 5.5 characters). So, what exactly do you need a supercomputer for, if
you're training on just 3 ish MB?
Not to mention- 2.8 MB might be OK to train a simple statistical model, but
for grammar induction? Oh, you can forget about that!
I don't see any results either- just some "excitement" about how the thing
"seems to be yielding positive results". So- where are they?
Sorry but I have to say that this article is really a bit rubbish.
________________
[1] Grammar induction is thee sub-field of machine learning that researches
the possibility of learning languages from examples.
[2] Oh, I don't know- try Colin de La Higuera, Grammatical Inference -
Learning Automata and Grammars. You won't find any source disagreeing on the
intractability issue.
- jnbiche 9y agoI had the exact same reaction to this article. Either they've poorly reported on the project, or the project investigator is hoping to make what would be a quantum leap in the state of the art. I am extremely skeptical that this will provide any kind of actionable disaster relief language materials at any time in the next 5 or even 10 years. It's fine and worthy to do this kind of research as basic research, but to market it (and fund it?) as some kind of "developing countries/disaster relief" project seems borderline deceptive.
- glup 9y agoRegarding your second point, #grantsmanship. Unfortunately it's quite hard to get funding for basic research without arguing it will help (=health intervention, diagnostics) or hurt (=help kill the bad guys, or at least manage them). Regarding your first point, this isn't so nutty. Bayesian approaches to PCFG induction are getting increasingly sophisticated. It's not grammar identification in the limit, it's coming up with a probabilistic linguistic representation that improves your predictions (all models are wrong, these are just more useful than having no expectations regarding linguistic structure).
- glup 9y agoOf note, children demonstrate pretty robust grammatical expectations for specific syntactic phenomena after hearing comparable number of utterances, with conflicting evidence regarding the availability or reliability of negative evidence. Of course the proofs you cite from formal work are unassailable (=math), but it's possible that "learning" tolerates more error than identification in the limit (see PAC learning for example), and that developing a facility with natural language has more to do with matching the distributions over strings from the adult grammar than inferring the correct grammar (identifying which utterances are in the language, vs. those that aren't). Child language induction is often used as the model for zero-resource natural language processing. This seems to be the direction Dr. Schuler is coming from: http://aclweb.org/anthology/C/C16/C16-1092.pdf http://aclweb.org/anthology/C/C16/C16-1092.pdf.
- YeGoblynQueenne 9y agoI'm not convinced about the "learning to match the distributions of strings, rather than the full structure of a language" bit. I'm not saying that you can only learn a language in full. For instance, I speak Greek natively and I know a bit of Italian. I don't have to know Italian perfectly to read books (and fumetti) and communicate in it. Then again, I already know a language so the problem of learning a new one is quite a different one from learning the first. The problem I have with statistical language learning, especially in children, is that, just because you can learn a distribution over strings doesn't mean you should, or that you do, as a human. I'll move the goal posts a bit now (but then again, you started talking about language learning in children) but an important ability of language users is that they can produce coherent utterances in context. Statistical language models can learn some structure and parameters, but I have yet to see any statistical learner that can do what humans do, know what it's saying and what is being said. This tells me that, although statistical learning may take us even say half-way to language learning, it won't get us all the way. I'm convinced that there's something much simpler, much easier (in terms of computational operations) that children -and adults- do when they learn language. We just haven't found out what it is, yet. Btw, limited human ability for introspection not withstanding, I can tell that when I try to understand a spoken or written utterance I definitely do something sort of probabilistic- I will sometimes even consciously ask myself what is more likely being said. But that doesn't tell me that language learning, or intelligence itself, is probabilistic- in the same way that just because I can walk I don't think of myself as a pair of legs with stuff on top (accurate as that description might be). Edit: upvote from me, for being the first person on HN that knows what I'm talking about when I'm talking about Gold's result etc. I was starting to think it was all a dream I had.
- bitL 9y agoDoesn't Bayesian approach work only on very simple spaces behaving linearly, i.e. having some nice stable fixed points they can converge to? I am surprised anyone is still using it for more advanced tasks...
- GolDDranks 9y agoMaybe you are talking about grammars of NON-natural languages or about the current state of research, which isn't there yet, but grammar induction is shown to be possible by children learning natural languages.