If you were to tell me curated domains is just a bootstrapping method and as the search engine evolves it will change, fine This makes me think of a possible a

If you were to tell me curated domains is just a bootstrapping method and as the search engine evolves it will change, fine This makes me think of a possible approach. Curate a giant set of domains that almost exclusively host high quality content. Crawl said domains. Use all of the crawled data as a training set to create a model with which to ascertain the quality of random Web pages from other domains. Then spider everything and run it against the model.