3 ms·
People were starting to crawl it. Surely it was huge, they were probably crawling bits they were interested in. I am absolutely positive there were people tryin
by gala8y 4y ago
People were starting to crawl it. Surely it was huge, they were probably crawling bits they were interested in. I am absolutely positive there were people trying to crawl a lot. It was a panic of kind, like with geocities. Still, would love to see what was preserved.
- joshu 4y agoI doubt they got very far. Most of the pages cut off very early since humans never went more than a page or two in. Maybe the new version wasn’t as good but when I left it wasn’t easy. Any at scale scraping was easy to detect since the site was so resource impoverished that it would have a performance impact.