2 ms·
If a legit user accesses the link through an <img> tag, the browser will send some telling headers. Accept: image/..., Sec-Fetch-Dest: image, etc. You can also
by kijin 11mo ago
If a legit user accesses the link through an <img> tag, the browser will send some telling headers. Accept: image/..., Sec-Fetch-Dest: image, etc.
You can also ignore requests with cross-origin referrers. Most LLM crawlers set the Referer header to a URL in the same origin. Any other origin should be treated as an attempted CSRF.
These refinements will probably go a long way toward reducing unintended side effects.
- Terr_ 11mo agoEven if we somehow guard against <img> and <iframe> and <script> etc., someone on a webforum that supports formatting links could just trick viewers into clicking a normal <a>, thinking they're accessing a funny picture or whatever. A bunch of CSRF/nonce stuff could apply if it were a POST instead... It may be more-effective to make the link unique and temporary, expiring fast enough that "hey, click this" is limited in its effectiveness. That might reduce true-positive detections of a bot that delays its access though.
- kijin 11mo agoIf it were my forum, I would just strip out any links to the honeypot URL. I have full control over who can post links to what URL, after all. You could use a URL shortener to bypass the ban, but then you'll be caught by the cross-origin referrer check.