3 ms·
This kind of comment is what I come to HN for, so thank you. I use OpenCalais to compute keywords for texts; where does it stand vs. [6] and LSA in general?
by bambax 14y ago
This kind of comment is what I come to HN for, so thank you.
I use OpenCalais to compute keywords for texts; where does it stand vs. [6] and LSA in general?
- riffraff 14y agoon a related note, I've been told once that OpenCalais actually uses a rule-based system, does anyone know if that's true?
- Dn_Ab 14y agoAt their scale I doubt they are using LSA or anything of similar complexity. My guess would be something like tf-idf coupled with any thing from naive bayes to logic programming for entity recognition. Final output would likely use some hand curated rules on how to combine the above with infromation from curated databases like dbepida. ________________ There is this thing called random projection based on the observation that in high dimensions, most direction are nearly orthogonal to each other. This is a wonderful observation because it allows us to write stupendously simple (and fast) algorithms that give good results for mathematically sound reasons. I strongly suggest anyone who is interested in LSA but wants something scalable, simpler to write and faster to look into Random Indexing.