9 ms·
SpaCy 3.0
- bratao 6y agoThank you Matthew, Ines, Sofie and Adriane for spaCy. It is a fundamental piece for me, both for work in Academia and in Industry.
- mark_l_watson 6y agoThanks to the SpaCy team! I spent a lot of time over about 20 years working on my own NLP tools. I stopped doing that and mostly now just use SpaCy (and sometimes Huggingface and Apple’s NLP models).
- Abishek_Muthian 6y agoHave you compared SpaCy with Apple's NLP on M1? I presume GPU is not being used by SpaCy on M1.
- t-vi 6y agoGiven that SpaCy uses PyTorch, that is being worked on.
- mark_l_watson 6y agoI have used both on M1, but compared them.
- stingraycharles 6y agoOk so I’ve evaluated spacy a few years ago, but nowadays we’re using huggingface’s transformers / tokenizers / etc to train our own language models + fine tuned models. I see there’s now transformer based pipeline support, how do the two relate? Phrased differently, how does spacy fit in with today’s world of transformers? Would it still be interesting for me?
- binarymax 6y agoI have lots of experience with both, and I use both together for different use cases. SpaCy fills the need of predictable/explainable pattern matching and NER - and is very fast and reasonably accurate on a CPU. Huggingface fills the need for task based prediction when you have a GPU.
- danieldk 6y agoHuggingface fills the need for task based prediction when you have a GPU. With model distillation, you can make models that annotate hundreds of sentences per second on a single CPU with a library like Huggingface Transformers. For instance, one of my distilled Dutch multi-task syntax models (UD POS, language-specific POS, lemmatization, morphology, dependency parsing) annotates 316 sentences per second with 4 threads on a Ryzen 3700X. This distilled model has virtually no loss in accuracy compared to the finetuned XLM-RoBERTa base model. I don't use Huggingface Transformers, but ported some of their implementations to Rust [1], but that should not make a big difference since all the heavy lifting happens in C++ in libtorch anyway. tl;dr: it is not true that tranformers are only useful for GPU prediction. You can get high CPU prediction speeds with some tricks (distillation, length-based bucketing in batches, using MKL, etc.). [1] https://github.com/tensordot/syntaxdot/tree/main/syntaxdot-transformers/src/models https://github.com/tensordot/syntaxdot/tree/main/syntaxdot-t...
- binarymax 6y agoInteresting. Did you start from a Distilled base model (like DistilRoBerta), or did you distill your fine-tuned model?
- danieldk 6y agoSorry for the late reply. I distilled from my own finetuned XLM-RoBERTa model.
- aabhay 6y agoIs there a standard template for creating a distilled model? I didn’t see a public hugging face implementation, just the models
- The_rationalist 6y agoI see that by default the trf model is roberta_base https://spacy.io/models/en#en_core_web_trf https://spacy.io/models/en#en_core_web_trf Is there an easy way to use xlnet (from transformers) for pos tagging, dep parsing, etc? Btw it would have been a smarter default as it scores more sota results on paperswithcode.com
- syllogism 6y agoYes you can easily train with xlnet instead of roberta-base --- just write the different model name in the config file (or pass a different string value, if doing it from Python). You can find an example config file here: https://github.com/explosion/projects/blob/v3/benchmarks/ner_conll03/configs/transformer.cfg https://github.com/explosion/projects/blob/v3/benchmarks/ner... I actually didn't see a performance improvement when using XLNet over roberta-base. I always wondered about this; ages ago I looked into it and I wasn't sure that the preprocessing details in the transformers version were entirely correct. Given very similar accuracies from XLNet and RoBERTa, I preferred RoBERTa for the following reasons: * I've never been able to understand the XLNet paper :(. I spent some time trying when it was released, but I just didn't really get it, not anything close to the level where I'd be able to implement it, anyway. * Standardising on BERT architecture has some advantages. If we mostly use BERT, we have a better chance of using faster implementations. Mostly nobody is training new XLNet models, whereas many new BERT models are being trained.
- cambalache 6y ago> spaCy is a library for advanced Natural Language Processing in Python and Cython.
- syllogism 6y agoI actually submitted this with (Python Natural Language Processing) after it, but it got edited away. I always find it hard to predict the preferred title style here...
- ZeroCool2u 6y agoSpaCy and HuggingFace fulfill practically 99% of all our needs for NLP project at work. Really incredible bodies of work. Also, my team chat is currently filled with people being extremely stoked about the SpaCy + FastAPI support! Really hope FastAPI replaces Flask sooner rather than later.
- gavinray 6y agoThere's some native SpaCy + FastAPI integration being created?! This sounds mindblowing, off to Google I go...
- thesehands 6y agoThe author of FastAPI https://twitter.com/tiangolo https://twitter.com/tiangolo is a Spacy employee
- tiangolo 6y agoYep, that's me, I work at Explosion (spaCy's home)!
- ZeroCool2u 6y agoBig fan my dude! While FastAPI is amazing, the docs for it are a work of art. I know a few people that have just used the FastAPI docs to learn what API's are and how they work, nevermind how use FastAPI itself.
- tiangolo 6y agoThanks for saying that! :)
- tiangolo 6y agoWith the new "spaCy project" it's easy to generate the boilerplate for many spaCy related projects, including a FastAPI app to serve models. But apart from that, there's actually not really much to integrate, as it all just works. Both libraries are actually (well ) designed to be decoupled. So that you can mix them independently with anything else. Note: I'm the creator of FastAPI, and also a developer at Explosion (spaCy's home).
- screye 6y agoI have been using Spacy3 nightly for a while now. This is game changing. Spacy3 practically covers 90% of NLP use-cases with near SOTA performance. The only reason to not use it would be if you are literally pushing the boundaries of NLP or building something super specialized. Hugging Face and Spacy (also Pytorch, but duh) are saving millions of dollars in man hours for companies around the world. They've been a revelation.
- JPKab 6y agoEverything in the above paragraph sounds like a hyped overstatement. None of it is. As someone that's worked on some rather intensive NLP implementations, Spacy 3.0 and HuggingFace both represent the culmination of a technological leap in NLP that started a few years ago with the advent of transfer learning in NLP. The level of accessibility to the masses these libraries offer is game-changing and democratizing.
- drran 6y agoCan you help, please? I want to use AI to translate (localize) messages for free software, in my case, Ukrainian language. My plan to improve quality of automated translation is to translate from similar languages in parallel, i.e. give a same message in English, Russian, Polish, and expect message in Ukrainian as output. Where I should start? Which libraries to use? How to connect them? How to train them?
- chartpath 6y agoI've been using LASER from Facebook Research via https://github.com/yannvgn/laserembeddings https://github.com/yannvgn/laserembeddings to accept multi-lingual input in front of the the domain-specific models for recommendations and stuff (that are trained on English annotated examples).
- zburatorul 6y agoThis sounds interesting. Can you share more please? It sounds like there is some multilingual input text on the basis of which you make recommendations, but I think you would have called that a search engine rather than recommender.
- pabs3 6y agoPlease note that Explosion does not like redistribution of SpaCy, they expect everyone to only use the builds they produce, so it would not be a good idea to package it for your favourite distro.
- dna_polymerase 6y agoPretty poor choice of license if they wanted me to care about their builds, tbh.
- pabs3 6y agoTheir concern is poor user experience when the documentation on the web doesn't match what versions are being redistributed by other folks.
- syllogism 6y agoI'm sorry that this conflicted with your plans, but I feel strongly that distributing Python libraries via system package managers such as apt is very bad for users. The pain is felt especially by users who are relatively new to Python, who will end up with their system Python in a confusing state that is difficult to correct. We of course encourage anyone to clone the repo or install from an sdist if they want to compile from source. In fact you can do the following: git clone https://github.com/explosion/spaCy cd spaCy make This will build you a standalone executable file, in the pex format, that only depends on your system Python and does not install any files into your system. You can copy this artifact into your bin and use it as a command-line application.
- t-vi 6y agoThat is sad, I would use SpaCy more if it had Debian packages (in particular in Debian). Python stuff packaged by Debian seems to work very well for me and has been for years.
- pabs3 6y agoAt $work we have internal packages of SpaCy and dependencies, not yet updated to 3.0 though. It was relatively easy to package using py2dsc from stdeb.
- pplonski86 6y agoIs there any framework similar to SpaCy or HugginFaces but for images?
- syllogism 6y agoI haven't had a situation to use it, but I think Kornia looks cool: https://github.com/kornia/kornia https://github.com/kornia/kornia
- langitbiru 6y agoSo with SpaCy 3.0, HuggingFace, do we still have a reason to use NLTK? Or they complement each other? Right now, I lost track of the progress in NLP.
- gillesjacobs 6y agoNLTK is showing its age. In my information extraction pipelines, the heavy lifting for modelling is done by SpaCy, AllenNLP, and Huggingface (and Pytorch or TF ofc). I only use NLTK since it has some base tools for low-resource languages for which noone has pretrained a transformer model or for specific NLP-related tasks. I still use their agreement metrics module, for instance. But that's about it. Dep parsing, NER, lemmatising and stemming is all better with the above mentioned packages.
- datameta 6y agoI wonder if it is sheer coincidence that SpaCy is pronounced the way the russian word "спасай" is. It means "rescue" (v.)
- magicalhippo 6y agoI stumbled over SpaCy when looking for something to extract key words and numbers from sentences, however it looked a bit daunting and/or overkill. Think recipes or similar, turning "take three tablespoons of sugar" into [3, 'tablespoons', 'sugar'] or similar. Should I give it another shot or are there libraries more suited for this than just plain regexp galore?
- teruakohatu 6y agoI did that years ago for some project. For recipes you can probably get away with regular expressions. But with Spacy you could tokenize the sentence, then tag each work with the Part of Speech it is, and then find patterns eg. Verb Number Noun Preposition Noun Then match the first noum against a list of measurements (tablespoon, teaspoon, tbsp...) and extract the rest of the components.
- SirHoki 6y agoThis is exactly what you want: https://github.com/facebook/duckling https://github.com/facebook/duckling
- magicalhippo 6y agoThat looks great, time to dip my feet in the Haskell pond then. Thanks!
- gillesjacobs 6y agoExcited for this release and I will start integrating this in my own information extraction pipelines immediately. Thanks, Explosion team, got your stickers on my notebook! The new configuration approach looks familiar to AllenNLP and that's great. Loose-coupling of model submodules with flexible config should be standard in NLP. I am happy that more libraries are integrating these concepts.
- Xenoamorphous 6y agoI think I read somewhere that spaCy was going to have named entity disambiguation at some point, with named entities having links to knowledge bases like Wikidata or DBpedia. That’s something that paid NER services but that I haven’t found in open source libs, and would be really interesting IMO.
- svlandeg 6y agoThere's a component for Entity Linking available in spaCy, but you have to train it yourself, as the use-cases (type of entities, type of knowledge base etc) can vary greatly. See more here: https://spacy.io/api/entitylinker https://spacy.io/api/entitylinker
- gyre007 6y agothere is also BLINK by Facebook https://github.com/facebookresearch/BLINK https://github.com/facebookresearch/BLINK
- polynomial 6y agoSuper excited to see improvement in NER accuracy in SpaCy 3.0.
- jcuenod 6y agoI'll add my hats off to to @ines and the spaCy team. It's super impressive. There's also a (free) orientation course I'd recommend at https://course.spacy.io/ https://course.spacy.io/
- liminal 6y agoI'm curious what sort of NLP use cases people are solving. How are people finding business value in these models and pipelines? We have looked at a number of uses and have found it hard to make a case for ROI. Wondering what's been working for folks.
- aabhay 6y agoAgreed. The larger and deeper the model, the more the use cases seem to become negative ROI. I think for business use there will have to be an effort to compress the architecture considerably
- hadsed 6y agoIt helps to be able to deliver value at a low bar. This stuff takes a lot of time and energy to improve upon, and it won't "just work". Search is usually a good place to start. Just make one little process a tiny bit better. Make sure you're tracking data very well, because if you're not already at a place to be making data-driven decisions you can't possibly take advantage of machine learning. After all that, just iterate until you've found a better problem to tackle with your newfound capabilities :)
- nojito 6y agoWe use it in healthcare to pluck out named entities that kicks off an email to the person mentioned.
- tsujp 6y agoThis sort of this could be used to create a "plain language" CLI right? So you could have the usual flags etc and then a separate version for less tech literate people that allows something like "list hidden files" (I know `ls -a` isn't particularly hard to remember I just need a contrived example).
- nocajar 6y ago@hannibal Would love to discuss how we could extend spacy as a powerful engine to also support processing from layouted documents. Just imagine how powerful it would be if you could throw a PDF document into it and it would preprocess it to text + layout, e.g. Paragraph and perform the next steps like extracting the right paragraph or date, adress, etc. I would love to provide/support that transformation. I have been doing similar things using rasa_nlu+spacy.
- yewenjie 6y agoI am not sure if SpaCy does it but is there some free and open-source framework capable of speech synthesis comparable to the level of AWS Polly or alike?
- suyash 6y agoHas anyone tried it on Raspberry Pi, will it work well?
- juliensalinas 6y agoThat's really cool to see how accuracy of pre-trained models is improving by simply switching to v3.0 I've been using the v3 nightly version for 2 months and it works like a charm. I'm now training models with v3 and using them in production without any issue. Great job!