6 ms·
In the case of collisions, you need to search for a bunch of possible rest-of-sentence phrases in the text. This is exactly the problem that your algorithm trie
by pjscott 15y ago
In the case of collisions, you need to search for a bunch of possible rest-of-sentence phrases in the text. This is exactly the problem that your algorithm tries to solve! If you apply it recursively, you get, essentially, trie matching with words instead of characters.
http://en.wikipedia.org/wiki/Trie http://en.wikipedia.org/wiki/Trie
- jodrellblank 15y agoThere is some bunch searching, but most of the document doesn't get searched, and the bits which get searched aren't searched for most of the keyphrases. Instead of: For phrase in keyphrases: If phrase in document: pass This For word in document: Some_phrases = get_keyphrases_starting[word] if document.next_5_words.find(some_phrases) Instead of searching 100,000 keyphrases in the whole document, this searches 10 rest-of-sentences in the 5 words following a mention of 'phospholipid', 18 rest-of-sentences in the 3 word following 'proteomerase' and nothing in the text about the lab temperature, and never touches most of the keyphrases at all for the document.