4 ms·
2gram corpus is not that easy to implement, but here is my implementation with POS and frequency sorting. http://buzzdomain.com http://buzzdomain.com
by aquarin 13y ago
2gram corpus is not that easy to implement, but here is my implementation with POS and frequency sorting.
http://buzzdomain.com http://buzzdomain.com
- rahimnathwani 13y agoNice! Small typo on home page: 'When you _clik_ the Search button'
- mikkom 13y agoIMHO 2-gram corpus is very easy to implement, what do you find difficult with it?
- Todd 13y agoI've done this before. You're right. The CPU intensive part is the word search itself. Building the combinations and finding adjacent words, which you've already done. The ngram piece is straightforward by comparison. Just compare the domain (with any hyphens removed) against a dictionary of concatenated ngrams. You'll have to veryify that any hyphens occur on word breaks afterwards, but it's a small price to pay for the simplicity of a string compare (or key lookup). You will probably want to load all of your ngram dictionaries into memory, though, or the search will be horribly slow.
- aquarin 13y agoI have already done this, but it is difficult to squeeze the running algorithm memory. The current POS solution uses only 400MB memory. Also, 2grams are not that good as collocations.