2 ms·

Announcing the Public Terabyte Dataset project

• adatta02 17y ago
  "This is a high quality crawl of top web sites, using AWS’s Elastic Map Reduce, Concurrent’s Cascading workflow API, and Bixolab’s elastic web mining platform." so two questions. what exactly is a "high quality" crawl? and who is going to be the judge of "top websites"?