2 ms·

Thanks for posting this. Good to know that using just vanilla Python is as viable for learning as using specialized frameworks.

• jventura 10y ago
  Oh, definitely you don't need much more than vanilla Python for playing around and test things. The only thing that you may need is a good tokenizer (i.e., to know where you should break the words), but the regex in [0] was good enough for what I needed. I was working with texts for three different languages (EN, DE and PT), and that is the reason I recommend statistical approaches, as they tend to be language independent. [0] https://github.com/joaoventura/WikiCorpusExtractor/blob/master/tokenizer.py#L47 https://github.com/joaoventura/WikiCorpusExtractor/blob/mast...
• syllogism 10y ago
  You might find these posts interesting: https://explosion.ai/blog/part-of-speech-pos-tagger-in-python https://explosion.ai/blog/part-of-speech-pos-tagger-in-pytho... https://explosion.ai/blog/parsing-english-in-python https://explosion.ai/blog/parsing-english-in-python These days I would say these articles are better indications of solving NLP problems with linear models -- tagging and parsing are less important than they used to be. Here's how I think about doing NLP with current neural network techniques: https://explosion.ai/blog/deep-learning-formula-nlp https://explosion.ai/blog/deep-learning-formula-nlp