4 ms·
https://git.sr.ht/robots.txt https://git.sr.ht/robots.txt > YandexBot is a dickhead, too aggressive > User-agent: Yandex > Disallow: / Hahaha.
by paulcarroty 4y ago
https://git.sr.ht/robots.txt https://git.sr.ht/robots.txt
> YandexBot is a dickhead, too aggressive
> User-agent: Yandex
> Disallow: /
Hahaha.
- n8ta 4y ago# Used for Alexa, I guess, who cares User-agent: Amazonbot Disallow: /
- deleted 4y ago[deleted]