4 ms·
They're really bad at purging outdated search results, I often see them trying to GET pages that haven't existed in 5+ years.
by nubinetwork 3y ago
They're really bad at purging outdated search results, I often see them trying to GET pages that haven't existed in 5+ years.
- Kye 3y agoIt isn't just a search engine. It's also a link monitor. Someone somewhere is linking those pages. It might not be a current user, but they still crawl it in case the person with those bad links signs up.