Oh, definitely you don't need much more than vanilla Python for playing around and test things. The only thing that you may need is a good tokenizer (i.e., to k

Oh, definitely you don't need much more than vanilla Python for playing around and test things. The only thing that you may need is a good tokenizer (i.e., to know where you should break the words), but the regex in [0] was good enough for what I needed. I was working with texts for three different languages (EN, DE and PT), and that is the reason I recommend statistical approaches, as they tend to be language independent. [0] https://github.com/joaoventura/WikiCorpusExtractor/blob/master/tokenizer.py#L47 https://github.com/joaoventura/WikiCorpusExtractor/blob/mast...