For many queries, it seems the top result is just Stackoverflow, Wikipedia or something simliar, then a couple of organic links, and then just junk. And on the

For many queries, it seems the top result is just Stackoverflow, Wikipedia or something simliar, then a couple of organic links, and then just junk. And on the second or third page, the results just stop. I have a feeling that at this point, it is a mixture of white- and blacklisting, and single-page classifiers based on heuristics or "AI". PageRank is surely in there, but probably not dominant. (Sidenote: I wonder if they secretly do ignore rel="nofollow" because so many links use it.) Funnily, I've used PageRank for some smaller search projects, and it still works very well if you use it in a vertical, say "educational resources" or "programming" or simliar. I just did a broad crawl, 3-4 steps from some known "good" seed URLs away, calculate PageRank and mix it in with Solr's classifiers.