14 ms·
Announcing SyntaxNet: The World’s Most Accurate Natural Language Parser
- jdp23 10y agoParsey McParseface is a great name.
- chubot 10y agoYeah I noticed that :) Is this a Simpsons reference? Can't quite place it.
- koide 10y agoGoogle for Boaty McBoatFace
- chipperyman573 10y agoNope, Boaty McBoatface[0] [0] http://www.theatlantic.com/international/archive/2016/05/boaty-mcboatface-parliament-lessons/482046/ http://www.theatlantic.com/international/archive/2016/05/boa...
- UK-AL 10y agoHavent heard about Boaty McBoatface?
- worldsayshi 10y agoI kind of feel that it dates further back?
- QuercusMax 10y agoDefinitely; I think it's originally from Friends (possibly in a slightly different form; Xey McXerson). Haven't tracked down the exact reference, but it's well over a decade old. http://www.duetsblog.com/2016/03/articles/trademarks/namey-mcnamerson-what-happens-when-you-crowdsource-names/ http://www.duetsblog.com/2016/03/articles/trademarks/namey-m...
- deleted 10y ago[deleted]
- dharbin 10y agoXey McXface is a canary for detecting naming contests within an organization.
- jdright 10y agoAnd also a good template for naming things! Thanks.
- throwanem 10y agoNot only that, but now you have a solid natural language parser to tell whether you need the optional 'e' before the 'y' in the first word!
- vinceguidry 10y agoSomeone write a Parsey McParseface-backed JSON API that does that.
- infogulch 10y agoA reference to the new polar ship that was publicly voted to be named 'Boaty McBoatface', but the science minister is too stuck up to roll with it[0]. [0]: https://news.ycombinator.com/item?id=11642618 https://news.ycombinator.com/item?id=11642618
- djsumdog 10y agoThey did name one of them Boaty..the tiny one / unmanned sensor drone. :-/ Vote for boaty! He's keepin hope afloat! http://shirt.woot.com/offers/vote-for-boaty http://shirt.woot.com/offers/vote-for-boaty
- TeMPOraL 10y agoI wonder what's with that X McXface thing. Yes, I've heard something about Boaty McBoatface (whatever that is), but is the Boatface thing original, or is it some kind of obscure American meme?
- ChrisClark 10y agoIt wasn't the original, and yes there is a sort of meme like it. Not always -face though. I've heard Beardy McBeardson, the pattern is usually X-y Mc-X-somthing.
- solipsism 10y agoSince Boaty McBoatface is a British ship, why would you assume it's an obscure American meme? Just kidding! As an American, I can say it's a decent assumption :D
- ohitsdom 10y agoI'm sure it's only a matter of time before someone puts this online in a format easily played with. Looking forward to that
- xigency 10y agoIt's already available here - https://github.com/tensorflow/models/tree/master/syntaxnet https://github.com/tensorflow/models/tree/master/syntaxnet echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh Input: Bob brought the pizza to Alice . Parse: brought VBD ROOT +-- Bob NNP nsubj +-- pizza NN dobj | +-- the DT det +-- to IN prep | +-- Alice NNP pobj +-- . . punct
- ohitsdom 10y agoI mean fully online, where I don't have to download and setup tensorflow.
- xigency 10y agoYes, probably a few days, unless you go through the effort to setup a web server. For other non-Parsey McParseface dependency parsers and POS taggers that are web accessible, see http://corenlp.run/ http://corenlp.run/ and http://nlp.stanford.edu:8080/parser/ http://nlp.stanford.edu:8080/parser/.
- astrange 10y agoDoes Google's have a better sense of humor than the 3 in this thread? They all fail on: Time flies like an arrow. Fruit flies like a banana.
- xigency 10y agoReally, the mechanism of all these parsers, including SyntaxNet, is the same in that they use statistical training data to set up a neural network. Here's a paper on the Stanford CoreNLP parser, which you can compare with Google's paper: http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf So, really all of the above parsers are weak in that they only output a single best parsing, when in reality sentences can have more than one valid structure, the principal example being the second sentence you've provided. I don't think Google's model has a better sense of humor than the others, no. I anticipate that they all have used relatively similar training data. However, there is probably a trivial way to get the second sentence to parse as Subject --- Verb --- Object Noun Verb Article Noun | \ | | | Fruit flies like a banana . and that is to provide training data with more occurrences of ... > N{Fruit flies} V{like} honey. > N{Fruit flies} V{like} sugar water. than occurrences of > A plane V{flies} PREP{like} a bird. The more sentences using simile that the parser finds, the less likely the neural net is to consider 'like' as a verb. It's also impacted by all of the uses of [flies like]. That's the nature of statistical language tools. The stock parser debuted here gives the same answer as CoreNLP, by the way. flies VBZ ROOT +-- Fruit NNP nsubj +-- like IN prep | +-- banana NN pobj | +-- a DT det +-- . . punct So much for Parsey McParseface's sense of humor.
- xigency 10y agoEvidence that this is the most accurate parser is here; the previous approach mentioned is a March 2016 paper, "Globally Normalized Transition-Based Neural Networks," http://arxiv.org/abs/1603.06042 http://arxiv.org/abs/1603.06042 "On a standard benchmark consisting of randomly drawn English newswire sentences (the 20 year old Penn Treebank), Parsey McParseface recovers individual dependencies between words with over 94% accuracy, beating our own previous state-of-the-art results, which were already better than any previous approach." From the original paper, "Our model achieves state-of-the-art accuracy on all of these tasks, matching or outperforming LSTMs while being significantly faster. In particular for dependency parsing on the Wall Street Journal we achieve the best-ever published unlabeled attachment score of 94.41%." This seems like a narrower standard than described, specifically being better at parsing the Penn Treebank than the best natural language parser for English on the Wall Street Journal. The statistics listed on the project GitHub actually contradict these claims by showing the original March 2016 implementation has higher accuracy than Parsey McParseface.
- wodenokoto 10y agoThe paper you mention is the world's best results and is macparseface with broader beam search and more hidden layers. This is an opensourcing of the March 2016 method (syntaxnet, note that in the paper there are results from several trained models) as well as a trained model that is comparable in performance but faster (macparseface). It is very hard to separate those two things from the way they write.
- weinzierl 10y agospaCy is another active open source (MIT) POS-tagger. In a previous discussion on HN[1] it was well received. There is a simplified educational 200 lines python version [2] of it. It claims 96.8% for the WSJ corpus. What am I missing here? [1] https://news.ycombinator.com/item?id=8942783 https://news.ycombinator.com/item?id=8942783 [2] https://spacy.io/blog/part-of-speech-pos-tagger-in-python https://spacy.io/blog/part-of-speech-pos-tagger-in-python
- deleted 10y ago[deleted]
- deanclatworthy 10y agoIt's really nice to have access to these kinds of tools. I am sure some folks from Google are checking this, so thank you. Analysis of the structure of a piece of text is the first step to understanding its meaning. IBM are doing some good work in this area. http://www.alchemyapi.com/products/demo/alchemylanguage http://www.alchemyapi.com/products/demo/alchemylanguage Anything in the pipeline for this project to help with classifying sentiment, emotion etc. from text?
- zappo2938 10y agoYes, we derive syntactic meaning from grammatical structure. It's one thing getting a machine to understand grammar and another to get a human to understand. If anyone is interested, Doing Grammar by Max Morenberg is an excellent source of knowledge about grammar.[0]He approaches grammar very systematically which is helpful if people want to train machines. [0] http://www.amazon.com/Doing-Grammar-Max-Morenberg/dp/0199947333 http://www.amazon.com/Doing-Grammar-Max-Morenberg/dp/0199947...
- jrgoj 10y agoNow for the buffalo test[1] `echo 'Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo' | syntaxnet/demo.sh' buffalo NN ROOT +-- buffalo NN nn | +-- Buffalo NNP nn | | +-- Buffalo NNP nn | | +-- buffalo NNP nn | +-- buffalo NN nn +-- Buffalo NNP nn +-- buffalo NNP nn [1]: https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...
- nihonde 10y agoHow does it handle the "translation" from that wiki page? Bison from Buffalo, which bison from Buffalo bully, themselves bully bison from Buffalo.
- jrgoj 10y ago> Input: Bison from Buffalo , which bison from Buffalo bully , themselves bully bison from Buffalo . Parse: Bison NNP ROOT +-- from IN prep | +-- Buffalo NNP pobj +-- , , punct +-- bison NN ccomp | +-- themselves PRP nsubj | +-- bully RB advmod | +-- from IN prep | +-- Buffalo NNP pobj +-- . . punct
- nihonde 10y agoInteresting. "bully RB advmod" I assume this is wrong, and probably triggered by the "ly" ending? Also, odd that it dropped the second clause and didn't see any trouble with the parsed results lacking a single verb. I guess it has to be able to handle fragments, etc.
- amelius 10y agoHow would you feed a sentence to a neural net? As I understand, the inputs are usually just floating point numbers in a small range, so how is the mapping performed? And what if the sentence is longer than the number of input neurons? Can that even happen, and pose a problem?
- wodenokoto 10y agoOne hot vectors. You build a dictionary of all words + one catch all for unknown words. Each word then has a position in a sparse vector. So for example : Yes = (0,0,1,0,0,...) No = (0,0,0,1,0,....) Convolutional and recurrent nets can handle inputs of arbitrary lengths.
- vhold 10y agoWhat wodenokoto said, and also look up "Word Embeddings", word2vec is a popular method. https://www.tensorflow.org/versions/r0.7/tutorials/word2vec/index.html https://www.tensorflow.org/versions/r0.7/tutorials/word2vec/... There's a bunch of blogs, tutorials, etc, around word2vec and other methods of generating vectors from a training set of words. Also, in the tensorflow models codebase where this syntaxnet code lives, there is an another tensorflow-using-method of generating word embeddings with demonstration code called Swivel https://github.com/tensorflow/models/tree/master/swivel https://github.com/tensorflow/models/tree/master/swivel
- zodiac 10y agoI think Syntaxnet is just using the NN to guide search instead of doing end-to-end parsing. That said, you can feed "sequences of stuff" into recurrent neural nets! See https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf https://papers.nips.cc/paper/5346-sequence-to-sequence-learn...
- jventura 10y agoAs someone who has published work in the NLP area, I always take claimed results with a grain of salt. With that said, I still will have to read the paper to know the implementation details, although my problem with generic linguistic approaches such as this one seems to be is that it is usually hard to "port" to other languages. For instance, the way they parse sequences of words may or may not be too specific to the English language. It is somewhat similar to what we call "overfitting" in the data-mining area, and it may invalidate this technique for other languages. When I worked on this area (up to 2014), I worked mainly in language-independent statistical approaches. As with everything, it has its cons as you can extract information from more languages, but, in general, with less certainties. But in general, it is good to see that the NLP area is still alive somewhere, as I can't seem to find any NLP jobs where I live! :) Edit: I've read it in the diagonal, and it is based on a Neural Network, so in theory, if it was trained in other languages, it could return good enough results as well. It is normal for English/American authors to include only english datasets, but I would like to see an application to another language.. This is a very specialized domain of knowledge, so I'm quite limited on my analysis..
- wodenokoto 10y agoIt's not particularly hard to port nlp to other languages when you use these methods. You are mostly limited by tagged corpora. Nlp is very much alive and well.
- wodenokoto 10y agoThey trained an expanded version of macparseface on CoNLL 09, which includes a bunch of languages and it performs very good too. Look at the March 2016 paper they cite.
- Animats 10y agoThis could lead to a fun WordPress plug-in. All postings must be parsable by this parser. Surprisingly, this thing is written in C++.
- zem 10y agoone interesting use i can think of is new improved readability scores that can take into account words that are common or uncommon depending on part of speech. (e.g. a text that used "effect" as a noun would be lower-level than one that used "effect" as a verb)
- Someone 10y agoFor those wondering: the license appears to be Apache 2.0 (https://github.com/tensorflow/models https://github.com/tensorflow/models)
- deleted 10y ago[deleted]
- egoegoego 10y agoLojban is unambiguous and interesting. The problem isn't the language, though. It's ego. When you can insert "I am the best" before every internet comment, the language used is not the problem. It's the ego.
- aaron-santos 10y agoI'd love to see the failure modes especially relating to garden path sentences. [1] [1] - https://en.wikipedia.org/wiki/Garden_path_sentence https://en.wikipedia.org/wiki/Garden_path_sentence
- xigency 10y agoStatistical models for parsing are almost guaranteed to fail at parsing garden path sentences.
- scriptle 10y agoDid I just read it as Skynet ?
- teraflop 10y agoThis is really cool, and props to Google for making it publicly available. The blog post says this can be used as a building block for natural language understanding applications. Does anyone have examples of how that might work? Parse trees are cool to look at, but what can I do with them? For instance, let's say I'm interested in doing text classification. I can imagine that the parse tree would convey more semantic information than just a bag of words. Should I be turning the edges and vertices of the tree into a feature vectors somehow? I can think of a few half-baked ideas off the top of my head, but I'm sure other people have already spent a lot of time thinking about this, and I'm wondering if there are any "best practices".
- gipp 10y agoThe typical approach is something like a tree kernel (https://en.wikipedia.org/wiki/Tree_kernel https://en.wikipedia.org/wiki/Tree_kernel). Looked into them briefly for a work project that never got off the ground, can't say too much about using them in practice.
- yolesaber 10y agoThis is actually really useful for a project I'm working on. I'm trying to detect bias in news sources using sentiment analysis and one of the problems I've run into is identifying who exactly is the subject of a sentence. Using this could be really helpful in parsing out the noun phrases and breaking them down in order to find the subject.
- writeslowly 10y agoI've been experimenting with Stanford's CoreNLP to identify named entities for analyzing RSS feeds and I was really impressed by how well it worked, having known nothing about the state of NLP research before I started. Especially things like being able to identify coreferences.
- yolesaber 10y agoYes, I've used that before. I'm currently using Textacy for python which is also really good. However, extracting the named entities from a sentence is still a ways off from determining what's the subject of the sentence, although it gives a good indication. Using NER + quality POS tagging and tree building should do the trick for me I think.
- deleted 10y ago[deleted]
- TeMPOraL 10y ago> Humans do a remarkable job of dealing with ambiguity, almost to the point where the problem is unnoticeable; the challenge is for computers to do the same. Multiple ambiguities such as these in longer sentences conspire to give a combinatorial explosion in the number of possible structures for a sentence. Isn't the core observation about natural language that humans don't parse it at all? Grammar is a secondary, derived construct that we use to give language some stability; I doubt anyone reading "Alice drove down the street in her car" actually parsed the grammatical structure of that sentence, either explicitly or implicitly. Anyway, some impressive results here.
- bobwaycott 10y agoI'm not sure about the claim on implicit lack of parsing structure. I read your example as who did what, where, in what. There must be some level of structural parsing and recognition so we understand it was Alice who drove in a car, that the car is owned by Alice, and that she, Alice, drove down the street, in her car. That we automatically understand all this seems to indicate some level of implicit parsing, right? Admittedly, it's been many years since I did any study of linguistics and language acquisition, so I'm pretty ignorant of the current state of knowledge here. Am I just layering my grammatical parsing atop an existing understanding that doesn't parse at all?
- TeMPOraL 10y agoI think observing how children learn their native language is pretty informative. They can speak and understand it very well, whether or not they were taught formal grammar at school. Personally, I know very, very little of Polish grammar (i.e. of my native language), and only little bit more of English grammar - and that is only because foreign language courses are pretty heavily grammar-laden. I'm not a linguist, but seeing how people a) can understand sentences that are grammatically malformed perfectly well, b) can easily derive meaning out of "sentences" stripped out of verbs ("I her dinner cinema Washington"), it seems to me that most of the work is being done by pattern-matching to known words and phrases. E.g. "drove down the street" is a kind of semantic unit on its own. Again, I'm not a linguist, but a lot of introspection as well as observing other people strongly suggest to me that humans do anything but parsing grammatical structures.
- jweir 10y agoParsey McParseface? Nice touch Google. https://github.com/tensorflow/models/tree/master/syntaxnet/syntaxnet/models/parsey_mcparseface https://github.com/tensorflow/models/tree/master/syntaxnet/s...
- gd2 10y agoRight, is this name for this product a sign that Google is becoming more human, and less engineering extreme sports minded?
- Fennhella 10y agoI think you're reading too much into it (If you're being serious). Doubt it had any other meaning other than the team just having fun making a quip at the Boaty Mcboatface poll for a ship's name.
- gd2 10y agoYes, I was being playful with language, rather than straight serious. But this topic invites that.
- kirykl 10y agoThis is from research not production, researchers in my experience are usually up for this type of fun
- matt4077 10y agoI'm pretty sure the borders between the two are fluid at google. I also wish middle management were a bit less afraid of such things. You can usually get cool ideas approved in the higher echolons (because a CEO doesn't have to fear too much, or possibly because these people actually are more imaginative). Names that have meaning, tell a story are incredibly useful for marketing, even if they sometimes sound unprofessional. Exp: 'Plan B' (morning after pill), 'CockroachDB', 'Virgin'. It's beyond me how anyone could have chosen the predictable public outcry instead of naming that boat Boaty McBoatface. That's probably the least offending name that ever resulted from an internet poll.
- the_decider 10y agoAccording to their paper (http://arxiv.org/pdf/1603.06042v1.pdf http://arxiv.org/pdf/1603.06042v1.pdf), the technique can also be applied to sentence compression. It would be cool if Google publishes that example code/training-data as well.
- deleted 10y ago[deleted]
- degenerate 10y agoI'd love to let this loose on the comments section of worldstarhiphop or liveleak and see what it comes up with...
- w_t_payne 10y agoCool - I reckon I'm going to try to use it to build a "linter" for natural language requirements specifications. (I'm a bit sad like that).
- vonsydov 10y agoOk google finally got something useful for AI out in open source.
- weinzierl 10y agoSay, I wanted to use this for English text with a large amount of jargon. Do have to train my own model from scratch or is it possible to retrain Parsey McParseface? How expensive is it to train a model like Parsey McParseface?
- wodenokoto 10y agoTraining the model is basically free. Obtaining annotated text for your domain can be quite expensive if you hire a trained linguist.
- escap 10y ago>Training the model is basically free if you have a few GPUs and don't pay electricity
- YeGoblynQueenne 10y agoMore like a thousand or so CPUs and a few hundred GPS.
- fpgaminer 10y agoOne of the projects I'd love to develop is an automated peer editor for student essays. My wife is an english teacher and a large percentage of her time is taken up by grading papers. A large percentage of that time is then spent marking up grammar and spelling. What I envision is a website that handles that grammar/spelling bit. More importantly, I'd like it as a tool that the students use freely prior to submitting their essays to the teacher. I want them to have immediate feedback on how to improve the grammar in their essays, so they can iterate and learn. By the time the essays reach the teacher, the teacher should only have to grade for content, composition, style, plagiarism, citations, etc. Hopefully this also helps to reduce the amount of grammar that needs to be taught in-class, freeing time for more meaningful discussions. The problem is that while I have knowledge and experience in the computer vision side of machine learning, I lack experience in NLP. And to the best of my knowledge NLP as a field has not come as far as vision, to the extent that such an automated editor would have too many mistakes. To be student facing it would need to be really accurate. On top of that it wouldn't be dealing with well formed input. The input by definition is adversarial. So unlike SyntaxNet which is built to deal with comprehensible sentences, this tool would need to deal with incomprehensible sentences. According to the link, SyntaxNet only gets 90% accuracy on random sentences from the web. That said, I might give SyntaxNet a try. The idea would be to use SyntaxNet to extract meaning from a broken sentence, and then work backwards from the meaning to identify how the sentence can be modified to better match that meaning. Thank you Google for contributing this tool to the community at large.
- tvural 10y ago"A large percentage of that time is then spent marking up grammar and spelling." As an aside, I don't think this is the optimal way to teach people how to write. What were the ideas in those papers? How were they organized? Do the student's arguments make sense? I think that's what most students spend most of their time thinking about when writing an essay, and it can be a bit demoralizing to see the teacher care just as much about whether the grammar was right. Most students can fix grammar mistakes relatively easily once they notice them anyway.
- seanmcdirmid 10y ago
- mdip 10y agoThis looks fantastic. I've been fascinated with parsers ever since I got into programming in my teens (almost always centered around programming language parsing). Curious - The parsing work I've done with programming languages was never done via machine learning, just the usual strict classification rules (which are used to parse ... code written to a strict specification). I'm guessing source code could be fed as data to an engine like this as a training model but I'm not sure what the value would be. Does anyone more experienced/smarter than me have any insights on something like that? As a side-point: Parsy McParseface - Well done. They managed to lob a gag over at NERC (Boaty McBoatface) and let them know that the world won't end because a product has a goofy name. Every time Google does things like this they send an unconscious remind us that they're a company that's 'still just a bunch of people like our users'. They've always been good at marketing in a way that keeps that "touchy-feely" sense about them and they've taken a free opportunity to get attention for this product beyond just the small circle of programmers. As NERC found out, a lot of people paid attention when the winning name was Boaty McBoatface (among other, more obnoxous/less tasteful choices). A story about a new ship isn't going to hit the front page of any general news site normally and I always felt that NERC missed a prime opportunity to continue with that publicity and attention. It became a topic talked about by friends of mine who would otherwise have never paid attention to anything science related. It would have been comical, should the Boaty's mission turn up a major discovery, to hear 'serious newscasters' say the name of the ship in reference to the breakthrough. And it would have been refreshing to see that organization stick to the original name with a "Well, we tried, you spoke, it was a mistake to trust the pranksters on the web but we're not going to invoke the 'we get the final say' clause because that wasn't the spirit of the campaign. Our bad."
- danieldk 10y agoCurious - The parsing work I've done with programming languages was never done via machine learning, Artificial languages (such a programming languages) are usually designed to be unambiguous. In other words, there is a 1:1 mapping from a sentence or fragment to its abstract representation. Natural language is ambiguous, so there is usually 1:N mapping from a sentence to abstract representations. So, at some point you need to decide which of the N readings is the most likely one. Older rule-based approaches typically constructed all readings of a sentence and used a model to estimate which reading is the most plausible. In newer deterministic, linear-time (transition-based) parsers, such ambiguities (if any) are resolved immediately during each parsing step. In the end it's a trade-off between having access to global information during disambiguation and having a higher complexity. So, naturally, the rule-based systems have been applying tricks to aggressively prune the search space, while transition-based parsers are gaining more and more tricks to incorporate more global information.
- hartator 10y ago> At Google, we spend a lot of time thinking about how computer systems can read and understand human language in order to process it in intelligent ways. There is 6 links in this sentence in the original text. I get it can help to get more context around it, but I think it's actually making the text harder to "human" parse. It also feels they have hired a cheap SEO consultant to do some backlink integrations.
- RichieAHB 10y agoIs this a joke? SEO consultant at Google? Backlinks? It's just a simple way to give it context, and I barely noticed the links. Can't work out if this is a joke.
- WWKong 10y agoAnyone know a tool that does Natural Language to SQL?
- escap 10y agohttp://quepy.machinalis.com/about.html http://quepy.machinalis.com/about.html http://kueri.me/product/ http://kueri.me/product/
- PaulHoule 10y agoMeh. This kind of parser isn't all that useful anyway. Parts of speech are one of those things people use to talk about language with, but you don't actually use them to understand language.
- corin_ 10y agoYou do subconciously. When you read "Dave punched John" you don't need to think "hmmm, is 'John' the object or the subject?" but if your brain hasn't figured out which is the object and which the subject you won't know who is hitting who. You don't need to be able to name or define parts of speech but you need to be able to parse them, or you won't understand anything.
- TeMPOraL 10y agoYeah, except are you sure that the way you parse a sentence maps 1:1 to what you learned as "parts of speech"? I think an equivalent understanding, one that seems to be more intuitive to the way my own brain works (if I can believe introspection), is that "punched" is to be read by default as "--punched-->", and "was/is punched by" pattern-matches to <--punched--". Arrow denotes who's punching whom.
- corin_ 10y agoThat's your brain taking a shortcut because you know that "punched" is a verb, and that a verb in that form is likely to be followed by a subject noun, etc. Even if you never study grammar and couldn't even answer the question "which is the verb in this sentence?" your brain has still learned to recognise the different types of word which is why you understand them. Disclaimer: I'm not a linguist, but by layman's standards I'm pretty confident.
- tuukkah 10y agoBest grammar is the one that best matches actual language use. What you describe is similar to how link grammars (try to) work: https://en.m.wikipedia.org/wiki/Link_grammar https://en.m.wikipedia.org/wiki/Link_grammar
- joosters 10y agoI don't see how a linguistic parser can cope with all the ambiguities in human speech or writing. It's more than a problem of semantics, you also have to know things about the world in which we live in order to make sense of which syntactic structure is correct. e.g. take a sentence like "The cat sat on the rug. It meowed." Did the cat meow, or did the rug meow? You can't determine that by semantics, you have to know that cats meow and rugs don't. So to parse language well, you need to know an awful lot about the real world. Simply training your parser on lots of text and throwing neural nets at the code isn't going to fix this problem.
- jonknee 10y agoThat's exactly why it's using a neural net and yes, a lot of text will fix this problem. The only reason why we know cats meow and rugs don't is by learning about cats and rugs. Throw enough training data at it and the parser will figure out what is meowing. An interesting example of this you can easily try for yourself is playing with Google's voice to text features--if you say silly things like "the rug meowed" you will have terrible results because no matter how clearly it can hear you its training data tells it that makes no sense.
- atdt 10y agoThis is actually a rather serious limitation of statistical approaches to language: they work best with utterances that have already been said, or with concepts that are already strongly associated in common speech. Such utterances may make up the bulk of what we say and write, but the remainder isn't gobbledygook. It contains most of the intimacy, poetry, and humor of interpersonal communication, all of which trade on surprise and novelty.
- rspeer 10y agoI'm glad they point out that we need to move on from Penn Treebank when measuring the performance of NLP tools. Most communication doesn't sound like the Penn Treebank, and the decisions that annotators made when labeling Penn Treebank shouldn't constrain us forever. Too many people mistake "we can't make taggers that are better at tagging Penn Treebank" for "we can't make taggers better", when there are so many ways that taggers could be improved in the real world. I look forward to experimenting with Parsey McParseface.
- scarface74 10y agoI started working on a parser as a side project that could parse simple sentences, create a knowledge graph, and then you could ask questions based on the graph. I used http://m.newsinlevels.com http://m.newsinlevels.com at level 1 to feed it news articles and then you could ask questions. It worked pretty well but I lost interest once I realized I would have to feed it tons of words. So could I use this to do something similar? What programming language would I need to use?
- zodiac 10y agoIs your work available somewhere? I would love to play with something like it.
- scarface74 10y agoIt was a hack in C# with no unit tests it never got past the conceptual stage.
- feral 10y agoI'd love to hear Chomsky's reaction to this stuff (or someone in his camp on the Chomsky vs. Norvig debate [0]). My understanding is that Chomsky was against statistical approaches to AI, as being scientifically un-useful - eventual dead ends, which would reach a certain accuracy, and plateau - as opposed to the purer logic/grammar approaches, which reductionistically/generatively decompose things into constituent parts, in some interpretable way, which is hence more scientifically valuable, and composable - easier to build on. But now we're seeing these very successful blended approaches, where you've got a grammatical search, which is reductionist, and produces an interpretable factoring of the sentence - but its guided by a massive (comparatively uninterpretable) neural net. It's like AlphaGo - which is still doing search, in a very structured, rule based, reductionist way - but leveraging the more black-box statistical neural network to make the search actually efficient, and qualitatively more useful. Is this an emerging paradigm? I used to have a lot of sympathy for the Compsky argument, and thought Norvig et al. [the machine learning community] could be accused of talking up a more prosaic 'applied ML' agenda into being more scientifically worthwhile than it actually was. But I think systems like this are evidence that gradual, incremental, improvement of working statistical systems, can eventually yield more powerful reductionist/logical systems overall. I'd love to hear an opposing perspective from someone in the Chomsky camp, in the context of systems like this. (Which I am hopefully not strawmanning here.) [0]Norvig's article: http://norvig.com/chomsky.html http://norvig.com/chomsky.html
- x5n1 10y agoI think you are right and I think in the human brain similar sort of hybrid processes happen in order to make sense of the world. In the end strong AI will look very much like a massive hybrid system and a conscious controller that takes and integrates that information into a understood model of the world.
- DonaldFisk 10y agoI largely agree with Chomsky. I think both approaches are needed for general AI: neural networks, or something like them, for low level perception and recognition; and symbolic AI for higher level reasoning. Without the symbolic layer, you can't be sure what's going on. Symbolic AI has been very closely guided by cognitive psychology. Artificial neural networks ignore neurophysiology, so even when they work, they tell us very little about how the brain works. I keep hearing claims that symbolic AI is the wrong approach for anything, and that it failed. Yet there were quite a few successes (expert systems, discovery learning, common sense reasoning, for example) before sources of funding dried up.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- vicaya 10y ago1. WordNet 2. ImageNet 3. SyntaxNet ... n. SkyNet
- syncro 10y agoDockerized version so you try without installing: https://hub.docker.com/r/brianlow/syntaxnet-docker/ https://hub.docker.com/r/brianlow/syntaxnet-docker/
- sourcd 10y agoWhat would it take to build something like "wit.ai" using SyntaxNet ? i.e. to extract "intent" & related attributes from a sentence e.g. Input : "How's the weather today" Output : {"intent":"weather", "day":"Use wit-ai/duckling", "location":"..."}
- deleted 10y ago[deleted]
- bertan 10y agoParsey McParseface <3
- instakill 10y agoWhat are some use cases for this for hobbyists?
- neves 10y agoShouldn't the title be renamed for "The World's Most Accurate Natural Language Parser For English"? It's impressive how Google's natural language features, since the simpler spell check, degrades when it work with languages different from English.
- mindcrash 10y agoAnyone planning (or already busy) training Parsey with one of the alternative Treebanks available from Universal Dependencies [1]? Would love to know your results when you have any :) I am personally looking for a somewhat reliable NLP parser which can handle Dutch at the moment. Preferably one which can handle POS tagging without hacking it in myself. [1] http://universaldependencies.org/ http://universaldependencies.org/
- zodiac 10y ago> It is not uncommon for moderate length sentences - say 20 or 30 words in length - to have hundreds, thousands, or even tens of thousands of possible syntactic structures. Does "possible" mean "syntactically valid" here? If so I'd be interested in a citation for it. Also, I wonder what kind of errors it makes wrt to the classification in http://nlp.cs.berkeley.edu/pubs/Kummerfeld-Hall-Curran-Klein_2012_Analysis_paper.pdf http://nlp.cs.berkeley.edu/pubs/Kummerfeld-Hall-Curran-Klein...