The part about a solid pipeline is very important. Recently I talked to someone who had a bad experience detecting phrases using gensim. I always have had a goo

The part about a solid pipeline is very important. Recently I talked to someone who had a bad experience detecting phrases using gensim. I always have had a good experience so I was curious. Turns out they were doing OCR on PDFs and feeding the result directly into gensim. As anyone with OCR experience knows, there is often a lot of noise which is definitely going to screw up gensim or any other NLP tool. The quality of your preprocessing is important!