3 ms·

After reading Steven Pinker's book The Stuff of Thought in which he describes verbs as the chassis of a sentence, I thought we would see various Natural Languag

After reading Steven Pinker's book The Stuff of Thought in which he describes verbs as the chassis of a sentence, I thought we would see various Natural Language Understanding algorithms that leveraged "verb classes". Beth Levin published a book called Verb Classes and Alternations which is effectively a database/index of a subset of verbs. Each verb takes one or two arguments and the arguments can appear on the LHS, RHS, or both (i.e. alternations). You can get a feel for them from a 2011 paper by Levin [1]. Normally you would start with a Parts of Speech algorithm but with the limited context of flights (international only?) you could focus only on a "to" clause and a "from" clause with the city/country found in the RHS of each clause. Resolving ambiguous cities could start with a simple ranking algorithm based on a list of International/Domestic destinations and their corresponding traffic level. NLU is hard but constrained domains like this are tractable, in my overly optimistic opinion. [1] https://web.stanford.edu/~bclevin/mpeva11vclass.pdf https://web.stanford.edu/~bclevin/mpeva11vclass.pdf

PaulHoule 6y ago
I think the top answer is right philosophically in terms of putting entity recognition first and predicate recognition later. Trying to do it with grammar is exactly why you get into the rabbit hole where you get mixed up about Old York and New York. There is that "New York City" that daddy doesn't live in no more, and the "New York State" I live in where you frequently find dirt roads that look passable on GPS and paper maps but aren't. It is an ontological problem, not a grammatical problem. You are right that "air traffic level" is key in curating your city and/or airport database. You would start with a list like https://en.wikipedia.org/wiki/List_of_busiest_airports_by_passenger_traffic https://en.wikipedia.org/wiki/List_of_busiest_airports_by_pa... and work down from the top. At some point you can put together a "proof" that it works for the top 20% of the airports that get 80% of the traffic and work up from there to cover 95% of the traffic. For any project like this you need to build a test set if not a training or evaluation set. If you are not willing to look at 1000 examples by hand then don't get started. People are willing to spend six months fighting with everything from procurement to device drivers to run the same models on the same digits that everybody else runs, but it is hard to get people to make evaluation sets. I can't guarantee that the people who make those sets will succeed, but it is certain that those who don't make them will fail.

PaulHoule 6y ago

I think the top answer is right philosophically in terms of putting entity recognition first and predicate recognition later. Trying to do it with grammar is exactly why you get into the rabbit hole where you get mixed up about Old York and New York. There is that "New York City" that daddy doesn't live in no more, and the "New York State" I live in where you frequently find dirt roads that look passable on GPS and paper maps but aren't. It is an ontological problem, not a grammatical problem. You are right that "air traffic level" is key in curating your city and/or airport database. You would start with a list like https://en.wikipedia.org/wiki/List_of_busiest_airports_by_passenger_traffic https://en.wikipedia.org/wiki/List_of_busiest_airports_by_pa... and work down from the top. At some point you can put together a "proof" that it works for the top 20% of the airports that get 80% of the traffic and work up from there to cover 95% of the traffic. For any project like this you need to build a test set if not a training or evaluation set. If you are not willing to look at 1000 examples by hand then don't get started. People are willing to spend six months fighting with everything from procurement to device drivers to run the same models on the same digits that everybody else runs, but it is hard to get people to make evaluation sets. I can't guarantee that the people who make those sets will succeed, but it is certain that those who don't make them will fail.