7 ms·
Flair: A simple framework for natural language processing
- roman030 8y agoIs research in commercial companies becoming a driving factor in today's scientific achievements?
- StreamBright 8y agoProbably. However you have to consider fields that are outside of the scope of commercial companies. Nuclear energy (both fission and fusion) is a good example. If you are referring to NPL than it is definitely true.
- cleansy 8y agoThat's the free market at work I would guess. Zalando seems to see NLP research as a valuable asset so they pay for projects like this plus they open source it assumably because of employer branding. I don't see anything wrong with that.
- pploug 8y agoZalando Research is an internal team of researches, their work is primarily shared with the research community through publications: https://research.zalando.com/welcome/mission/publications/ https://research.zalando.com/welcome/mission/publications/ In the case of Flair, research led to a reference implementation, which was then matured through internal use and open sourced to further mature it and get external feedback. While employer branding is a nice benefit, it is a positive side-effect, not the motivation in itself :)
- bpicolo 8y agoIt always has been. The transistor was invented at Bell Labs in 1947, kicking off the entire modern world of computing.
- sweezyjeezy 8y agoI'd be wary of being dazzled by the performance metrics here. I've been disappointed in the past using out of the box language models on data that they weren't trained on, e.g. SpaCy. I feel like people put too much emphasis on trying to get the high score on benchmark datasets, and they're overtraining to those particular domains. For example, try using named entity models trained on CoNLL (newspaper articles) on free text (e.g. tweets, or text from application forms), you generally get pretty bad results. When the domain is different, I've even seen them screw up basic things like times and dates, where regexes will suffice. If you're using it for newspaper articles you're sorted, if you're not, the performance metrics here are probably not all that meaningful.
- sqrt17 8y agoHere's the thing: the authors, and most people in the NLP community (as opposed to people who can use off-the-shelf tools and nothing more) know how to get better performance on other domains. It's just that "it requires some adjustments and manual labour" doesn't add anything to the "deep learning solves the problem" narrative that is predominant in research articles and most blog posts on the topic. On the other hand, you can bet that actual practice in Zalando (the authors are all from Zalando's reasearch lab) involves more regexes and retraining models on proprietary datasets and less using off-the-shelf models and hope they stick. No one claims either that you can solve every Vision problem with a model trained on ImageNet - you'd do transfer learning, or for non-understanding problems (estimating colors and contrast or anything else that's unrelated to objects in the image) you'd use something else that doesn't involve deep learning models at all.
- sweezyjeezy 8y agoRight, but my point is - once you start needing to add ad-hoc retraining, or regex hacks, it's not clear to me that shaving a point off baseline f1 scores is really all that relevant anymore.
- yorwba 8y agoYou'd have to do the same modifications to the baseline models to adapt them to a different domain. If they managed to shave off percentage points on a large number of benchmarks, then it's likely that using their models will also help you with the task you care about.
- daolf 8y agoVery great and elegant API, look promising. Has anyone tried it? How does it stand against NLTK, Spacy or gensim ?
- sqrt17 8y agoIt solves different problems. flair doesn't include parsing, Spacy doesn't support embeddings, gensim doesn't do tagging or parsing at all but contains the most practical word2vec implementation. NLTK is nice for learning, but don't use it in production unless you're ready to reimplement things in a more efficient way when parts start falling off. The message is - again - learn about the mental framework, not individual tools, to understand where each strengths are and what the gaps are in between them. Or choose a problem, find the best tool for that problem and get progressively better at the tool(s) that helps you with most of your problems.
- deleted 8y ago[deleted]
- vstik 8y agoHow does it compare to managed NLP services like Google's Cloud Natural Language API and AWS's Comprehend?
- sqrt17 8y agoGoogle and Amazon have proprietary datasets for important sub-tasks (e.g. recognizing "consumer good" entities, or more accurate sentiment recognition, or supporting other languages better) that are not available to the public. In other words, if your problem looks like one of the benchmarking tasks in NLP research (e.g. recognizing persons and locations in fluent text) you can expect good performance out of open source tools. If you go beyond that, you have to concoct your own dataset and/or use proprietary cloud services.
- dhairya 8y agowould you mind linking to ones you are aware of. it would be super helpful.
- plagtag 8y agoI recently tried out flair and deployed it myself on aws. Works nice. However, if you need heavy duty batch inference i guess you need to use a gpu instance or so ...
- dajonker 8y agoI like the way Zalando is doing tech. They release (and maintain!) tons of open source stuff in several domains.
- duncanawoods 8y agoCan someone explain what the expected behaviour is with punctuation and Named Entity Recognition? Is there an assumption that punctuation is preprocessed in some form? I'm a noob but it's not what I expect - periods change what is extracted in inconsistent ways. e.g. "I love Berlin." -> "Berlin." "I love Berlin ." -> "Berlin" "George Washington loves Berlin." -> "George Washington" "George Washington loves Berlin ." -> ["George Washington", "Berlin"]
- slx26 8y agoIf you go to their first tutorial, "Tutorial 1: Basics", you will see this comment in the code: "# Make a sentence object by passing a whitespace tokenized string" In that simple example you posted, they already did the tokenization manually, as it's pretty trivial. But yes, in many cases, you have preprocessors that do the tokenization. In some libraries, you actually have a class/object_type for tokens, but it's pretty common to just preprocess and take every space as a token separator. In some contexts and cases, it's possible to see tokens like "social_network", where multiple words are considered a single token. In that first tutorial, they also mention they have a tokenizer if you need it: "In some use cases, you might not have your text already tokenized. For this case, we added a simple tokenizer using the lightweight segtok library." So for your example you would simply run the tokenizer first, then the named entity recognition. EDIT: apparently you can do this directly: "sentence = Sentence('The grass is green.', use_tokenizer=True)"
- duncanawoods 8y agoThanks. I assumed it was French style punctuation with a space before the period rather than space tokenization!
- yorwba 8y agoFrench doesn't put spaces before single-part punctuation like periods or commas.
- cjg_ 8y agoIt mentions in the tutorial that you can input space tokenized sentences directly or have it tokenize it for you, see https://github.com/zalandoresearch/flair/blob/master/resources/docs/TUTORIAL_1_BASICS.md#tokenization https://github.com/zalandoresearch/flair/blob/master/resourc....
- elyase 8y agoThe closest alternatives in this space would be allennlp [1], the recently released pytext [2] and spacy [3]. pytext's authors wrote some comparison on the accompanying paper [4] and this GitHub issue [5]. [1] https://github.com/allenai/allennlp https://github.com/allenai/allennlp [2] https://github.com/facebookresearch/pytext https://github.com/facebookresearch/pytext [3] https://spacy.io https://spacy.io [4] https://arxiv.org/pdf/1812.08729.pdf https://arxiv.org/pdf/1812.08729.pdf [5] https://github.com/facebookresearch/pytext/issues/110 https://github.com/facebookresearch/pytext/issues/110
- mkl 8y agoDo you know if any of these can be used for text prediction? (I.e. guessing what the next word/token will be.)
- be_erik 8y agoMarkov chains can be used to do type ahead prediction. It's likely what the iOS uses for their predictive keyboard. https://en.wikipedia.org/wiki/Markov_chain https://en.wikipedia.org/wiki/Markov_chain
- mkl 8y agoYes, there are plenty of methods, and I have a couple implemented, but an off-the-shelf one from a cutting edge library would likely be better.
- edraferi 8y agoIt's gonna be hard to get an "off the shelf" model for text prediction, because the upcoming text depends on the author, topic, and other context. You can probably find some decent pre-trained models to get started, but you'll need to customize them for your application to get good results.
- mkl 8y ago
- Nimitz14 8y agoNice! Was using stanford POS tagger which is both bad in quality and sooo slow in execution. Looking forward to trying this out.
- EmilStenstrom 8y agoFlair 0.4 was released just 14 days ago, and contiains LOTS of improvements for a point release: https://github.com/zalandoresearch/flair/releases https://github.com/zalandoresearch/flair/releases
- tgrzinic 8y agoWhat are the main advantages of Flair over Spacy? Is it easy to add a new language in Flair? In Spacy adding a language looks pretty straightforward.