3 ms·
Actually I was able to search URLs with very little ram across the entire collection as they have a series of indexes you can download. In theory someone could
by th0ma5 6y ago
Actually I was able to search URLs with very little ram across the entire collection as they have a series of indexes you can download.
In theory someone could do something similar with terms, or you could first use URLs to filter the text size you download into Elastic or Solr and do your own custom search that way.
The indexes are really neat though, I highly recommend playing with them.
- thunderbong 6y agoCan you suggest where I can get them from?
- th0ma5 6y agohttps://index.commoncrawl.org/ https://index.commoncrawl.org/