3 ms·
I believe I've seen this type of crawling in action in request logs. For example, Yahoo might try to request "news.ycombinator.com/user?id=britney_spears", eve
by sam_in_nyc 18y ago
I believe I've seen this type of crawling in action in request logs. For example, Yahoo might try to request "news.ycombinator.com/user?id=britney_spears", even though it's not linked to from anywhere.