You may want to have a look at these as they have similar goals to yours: http://openwebindex.eu/en/ http://openwebindex.eu/en/ https://deusu.org https://deus

You may want to have a look at these as they have similar goals to yours: http://openwebindex.eu/en/ http://openwebindex.eu/en/ https://deusu.org https://deusu.org Sourcecode for deusu.org is on: https://github.com/MichaelSchoebel/DeuSu https://github.com/MichaelSchoebel/DeuSu OpenWebIndex is only in the idea stage as far as I know. Deusu has been running for over a year and already has about 1.2 billion pages in their index.