3 ms·
That would be a different bot, one written specifically for your site. Mainly we're discussing the dumb ones that just crawl all possible http links
by inigyou 2mo ago
That would be a different bot, one written specifically for your site. Mainly we're discussing the dumb ones that just crawl all possible http links
- marginalia_nu 2mo agoNo I think this is the same case. It seems to just be following local links on the SERP. From a search for jack parsons, you can find hyperlinks to the sorts of requests it's making.