3 ms·
commoncrawl.org Our public web dataset goes back to 2008, and is widely used by academia and startups.
by ccgreg 10mo ago
commoncrawl.org
Our public web dataset goes back to 2008, and is widely used by academia and startups.
- pdimitar 10mo agoI always wanted to ask: - How often is that updated? - How current is it at any point in time? - Does it have historical / temporal access i.e. be able to check the history of a page a la The Internet Archive?
- ccgreg 10mo ago- monthly - it's a historical archive, the concept of "current" is hard to turn into a metric - not only is our archive historical, it is included in the Internet Archive's wayback machine.