3 ms·

For anyone who has used this, how does this compare to TextBlob or Gensim?

• syllogism 12y ago
  TextBlob is a wrapper around NLTK and Pattern. Those libraries don't use very sophisticated statistics, so in 2013 I wrote a small Python POS tagger for TextBlob, which performs much better: https://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/ https://honnibal.wordpress.com/2013/09/11/a-good-part-of-spe... spaCy's POS tagger works like the one in the blog post, but it's implemented in Cython, and has some extra features. Pattern has some nice morphological processing features. I don't do morphological generation, for instance, and I haven't hooked up the morphological analysis to the Python API yet. TextBlob also gives you a few extra bits and pieces, like a wrapper of the Google translate API. I'm really targeting the situation where you want to build a product around some NLP. In this use-case, you need the NLP to be fast, you need it to be as accurate as anyone knows how to make such a system, and you need it to be entirely in your control. As far as GenSim goes: it's good. It does different things from spaCy, though --- topic modelling, etc. It would be nice to interoperate between the two libraries. I have no plans to implement topic models.
  • Radim 12y ago
    ...and I have no plans to add NLP tools in gensim. The connection between gensim and tokenizing/tagging/parsing libs is intentionally loose and flexible. I'm a fan of "do one thing, do it well". Having said that, it would be great to facilitate "spaCy + gensim" pipelines for users. For example, the "word vector representations" can be trained easily with gensim, on arbitrary user-specified corpora, whereas spaCy loads something pre-trained, in a specific format. Maybe room for some interoperability there?