3 ms·
Take a look at the Google book corpus. I used it several years ago by renting the largest disk/RAM server Hetzner rents and collecting the most common 1gram, 2g
by mark_l_watson 10y ago
Take a look at the Google book corpus. I used it several years ago by renting the largest disk/RAM server Hetzner rents and collecting the most common 1gram, 2grams,...5grams in the book corpus. Many classification, summarization, etc. systems that rely on simple bag of words approaches can be improved by including ngrams. For a simple example, to label sentiment of text, considering 2grams lets you correctly handle 'not good', where considering one word at a time, ignoring word order, etc. is not so good.