3 ms·
I wish web.archive.org had an index by someone like common crawl. There is lots of great stuff on archive.org
by mycall 5y ago
I wish web.archive.org had an index by someone like common crawl. There is lots of great stuff on archive.org
- wumpus 5y agoweb.archive.org has a CDX index, similar to Common Crawl. Since I use both of these archives together, I wrote this code to iron out the differences between them: https://github.com/cocrawler/cdx_toolkit https://github.com/cocrawler/cdx_toolkit
- kevinsundar 5y agoHey! I was using your tool a couple months ago. It was super helpful for my project.
- wumpus 5y agoThanks! I rarely hear from users, great to hear from you!
- kevinsundar 5y agoThey do and its better than common crawl's by my testing.