21 ms·
Information Retrieval papers you need to read
- mahmud 18y agoAnon84, you have been churning out some good links lately. What is it? hack mode? beautiful isn't it? :-) [sorry, had to make a private remark publicly, to someone I don't know because HN doesn't have messaging. just had to acknowledge that Anon84, and others like him, has been keeping the place more "hackish" and less TechCrunch-fluff.]
- deleted 18y ago[deleted]
- Anon84 18y agoThanks for the support. I've been spending a lot more time lately reading up on this type of stuff for a huge scraping/data mining project. My background is mostly in Physics so depending on what I happen to be working on at any given moment there's always some catching up to do. Anything particularly interesting gets posted here for others to enjoy and comment. :)
- mahmud 18y agoI'm doing the same right now. Massive scraping plus "gisting" or document summarization. You're pretty much on the right track; half of those papers are industry standards (my browser marked them as "visited" automatically :-)
- GeneralMaximus 18y agoSo which of those would be a good starting point for someone who has no idea about IR?
- mahmud 18y agostart with LSA: laten semantic indexing, if you want information gisting/summarization. i'm not worried about IR because I use a good search-engine library: Montezuma, and it's in Common Lisp. It's Java clone is called Lucene :-)
- GeneralMaximus 18y agoA Google search gave me this: http://www.cs.utk.edu/~lsi/ http://www.cs.utk.edu/~lsi/ Is that what you're talking about?
- wheels 18y agoHonestly, I don't think LSI is a good place to start. It'd be better to start a few steps back with basic IR algorithms and data structures like tries, inverted indexes, stemming, string search algorithms like KMP and Boyer-Moore. When I got started with IR I used Baeza-Yate's "Modern Information Retrieval"; Manning's new "Introduction to Information Retrieval" looks great, though I haven't gotten my hands on a copy yet. Here are the Wikipedia pages on the concepts above: http://en.wikipedia.org/wiki/Trie http://en.wikipedia.org/wiki/Trie http://en.wikipedia.org/wiki/Inverted_index http://en.wikipedia.org/wiki/Inverted_index http://en.wikipedia.org/wiki/Stemming http://en.wikipedia.org/wiki/Stemming http://en.wikipedia.org/wiki/Knuth–Morris–Pratt_algorithm http://en.wikipedia.org/wiki/Knuth–Morris–Pratt_algorithm http://en.wikipedia.org/wiki/Boyer–Moore_string_search_algorithm http://en.wikipedia.org/wiki/Boyer–Moore_string_search_algor...
- smidwap 18y agoBruno, we've got to talk. I go to IU and you seem like a particularly interesting individual. I'll shoot you an email soon.
- dbul 18y agoGood choice. It is most definitely an underrated school. I miss the burrow.
- wheels 18y agoA couple other seminal favorites: Authoritative Sources in a Hyperlinked Environment http://www.cs.cornell.edu/home/kleinber/auth.pdf http://www.cs.cornell.edu/home/kleinber/auth.pdf Probabilistic Latent Semantic Analysis http://www.cs.brown.edu/~th/papers/Hofmann-UAI99.pdf http://www.cs.brown.edu/~th/papers/Hofmann-UAI99.pdf Kleinberg, author of the first paper, is quite possibly my favorite computer scientist. He's been on the leading edge of a lot of largely disjoint areas of research on the information structure of the web and almost always drops in some really smart stuff. He seems to hit new fields and drop in some big ideas and move on, so a lot of the more concretely applicable stuff comes in refinements of his work, but sometimes I feel like every time I cross into a new problem domain he's already written three papers on it. Beyond that, he writes in a very readable style, which is frustratingly rare for top-notch computer scientists.
- gfodor 18y agoThe "rebel king" is also a phenominal lecturer and a favorite among all cornell CS students.
- ananthrk 18y agoAll good links. Thanks for sharing. Another list of books for IR http://researchonsearch.blogspot.com/2005/12/information-retrieval-textbooks.html http://researchonsearch.blogspot.com/2005/12/information-ret...
- deleted 18y ago[deleted]
- alexitosrv 18y agoAlso you guys surely you would like this book too: http://www-csli.stanford.edu/~hinrich/information-retrieval-book.html http://www-csli.stanford.edu/~hinrich/information-retrieval-... http://nlp.stanford.edu/IR-book/pdf/irbookprint.pdf http://nlp.stanford.edu/IR-book/pdf/irbookprint.pdf Is a very readable and comprehensive introduction of the field. Besides it is recent and covers in a depth several current hot topics. here is a review the book http://glinden.blogspot.com/2009/02/book-review-introduction-to-information.html http://glinden.blogspot.com/2009/02/book-review-introduction...