4 ms·
That reminds me of something I was doing recently, really rough corpus analysis, trying to see how much text coverage the words in the NGSL give you. Honestly t
by 50CNT 10y ago
That reminds me of something I was doing recently, really rough corpus analysis, trying to see how much text coverage the words in the NGSL give you. Honestly thought it'd take me a couple of weeks to do.
Got into NLTK, used the built in sentence tokenizer, word tokenizer, then wordnet POS tagging to remove proper nouns, added some more cleanup code, and I had something passable within two days.
Now at this point I couldn't write a POS tagger to save my life, but it was cool seeing code you wrote over two evenings run over 30k books just like that (which still took a week, but ah well).