4 ms·
As someone else alluded to, this is a task for multiple models. Fortunately, there are a lot of great NLP libraries that combine multiple pre-trained language m
by calebkaiser 6y ago
As someone else alluded to, this is a task for multiple models. Fortunately, there are a lot of great NLP libraries that combine multiple pre-trained language models into a single pipeline you can interface with, like Stanza. From their docs, their vanilla pipeline breaks down the sentence "Barack Obama was born in Hawaii. He was elected president in 2008." as :
('Barack', '4', 'nsubj:pass')
('Obama', '1', 'flat')
('was', '4', 'aux:pass')
('born', '0', 'root')
('in', '6', 'case')
('Hawaii', '4', 'obl')
('.', '4', 'punct')
It should be very easy to deploy Stanza's pipeline as an API endpoint. Here is an example of such a NLP-library-as-API endpoint, albeit with Hugging Face's Transformers, deployed via Cortex: https://github.com/cortexlabs/cortex/blob/master/examples/pytorch/sentiment-analyzer/predictor.py https://github.com/cortexlabs/cortex/blob/master/examples/py...
- hiddencost 6y agoA language model is a model that predicts the probability of a given text, that is all. It should not be conflated with other types of NLP tasks like part of speech tagging. I'm guessing the popularity of transformer based models, which are built around the LM task and then adapted to other donations, is leading to this confusion.