34 ms·

I'm guessing he has a bunch of restaurant menus that he needs to read and categorize the items into appetizers, entrees, dessert, etc. etc. The problem is that

by jbob2000 9y ago

I'm guessing he has a bunch of restaurant menus that he needs to read and categorize the items into appetizers, entrees, dessert, etc. etc. The problem is that he'll need 50,000 menus to train the ML model on, and then another 50,000 to verify it. He'll also run into the problem of restaurants sometimes categorizing entrees as appetizers (is a caeser salad an entree or an appetizer?) so the NLP portion will be especially difficult. What about dish names that are in other languages? I could go on... Tough problem!

misiti3780 9y ago
I would recommend checking out Mechanical Turk
- vorotato 9y ago
  Yes humans have an advanced neural network that comes trained on all kinds of tasks.
PaulHoule 9y ago
You hit the nail on the head; for many straight information extraction problems, the universe of documents you want to extract could be too small to learn a model from -- or for that matter, for conventional methods of model evaluation to apply. (You want to extract data from all the items, not prove you reached a certain level of accuracy on a sub-sample of them) One approach is https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/ where you can throw in a great amount of unlabeled data and build an internal representation that models the data well enough that you can train something that works like an HMM or CRF with a tiny amount of labeled data. If you are willing to do something rule-based, I've used https://en.wikipedia.org/wiki/Case-based_reasoning https://en.wikipedia.org/wiki/Case-based_reasoning to organize the work in annotating corpuses. Often I can prove that a certain rule set covers X% of the cases, then add a rule to do X+epsilon% until the results are "good enough". Feel free to click on my profile link and send me a message if you want to chat more.

misiti3780 9y ago

I would recommend checking out Mechanical Turk

vorotato 9y ago
Yes humans have an advanced neural network that comes trained on all kinds of tasks.

PaulHoule 9y ago

You hit the nail on the head; for many straight information extraction problems, the universe of documents you want to extract could be too small to learn a model from -- or for that matter, for conventional methods of model evaluation to apply. (You want to extract data from all the items, not prove you reached a certain level of accuracy on a sub-sample of them) One approach is https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/ where you can throw in a great amount of unlabeled data and build an internal representation that models the data well enough that you can train something that works like an HMM or CRF with a tiny amount of labeled data. If you are willing to do something rule-based, I've used https://en.wikipedia.org/wiki/Case-based_reasoning https://en.wikipedia.org/wiki/Case-based_reasoning to organize the work in annotating corpuses. Often I can prove that a certain rule set covers X% of the cases, then add a rule to do X+epsilon% until the results are "good enough". Feel free to click on my profile link and send me a message if you want to chat more.