2 ms·
I thought about doing something like that, but then I realised: what if someone linked to the trap URL it from another site and a crawler followed that link to
by johneth 2y ago
I thought about doing something like that, but then I realised: what if someone linked to the trap URL it from another site and a crawler followed that link to the trap?
You might end up penalising Googlebot or Bingbot.
If anyone knew what that trap URL did, and felt malicious, this could happen.
- CodesInChaos 2y agoA crawler could easily avoid that by fetching the target domain's robots.txt before fetching the link target. However a website could also embed the honeypot link in an <img> tag and get the user banned when their browser attempts to load the image.