I concur with nkurz here. Inverted index are the basis for most full-text search engines and they can handle this quite well. Another issue with searching a l

I concur with nkurz here. Inverted index are the basis for most full-text search engines and they can handle this quite well. Another issue with searching a large forum like HN though can be keeping the index up-to-date through crawling new posts constantly to add the terms they contain to the inverted index. However, you can usually build a simple script that adds news posts URLs to a list of priority crawl targets. Free HTDig (http://www.htdig.org/ http://www.htdig.org/) for instance could do the task pretty well and run on a budget hosting plan described above. It can index up to 100ks of documents easily and may require disk space upgrades for large sites but not much more. There are many other inverted index search engines available and some now easily index without crawling through XML feeds from your CMS or forum software.