I like the project's goal but as techies, we inevitably want to understand the technical details and how it helps (or handicaps) the search results in compariso

I like the project's goal but as techies, we inevitably want to understand the technical details and how it helps (or handicaps) the search results in comparison with Google. For example, the project's data sources[1] says that the bulk of data comes from The Common Crawl. It looks like the CC is ~150 TB of data[2]. I'm not familiar with google.com internals but various sources estimate that their proprietary crawl dataset is more than a petabyte. (A googler could chime in here with more accurate data.) So it's not as simple as the algorithm for Common Search being "more fair" than the algorithm for Google Inc. The underlying dataset in terms of quantity, recency, rules for the robot, etc all affect the algorithm. This is not a criticism of the project. It is my attempt to understand what is not obvious on the surface level. [1]https://about.commonsearch.org/data-sources https://about.commonsearch.org/data-sources [2]http://commoncrawl.org/2015/12/november-2015-crawl-archive-now-available/ http://commoncrawl.org/2015/12/november-2015-crawl-archive-n... (I'm can't tell if each archive of MM/YYYY is cumulative or an addendum.)