4 ms·
The point of the post was that you need to add robots.txt (or similar) because it's a thing, and also tell us where they are.
by wseqyrku 4mo ago
The point of the post was that you need to add robots.txt (or similar) because it's a thing, and also tell us where they are.
- Geezus_42 4mo ago> add a robots.txt Which bots will then ignore.
- Ferret7446 4mo agoI didn't expect I'd get to use this so soon. Obligatory canned advisory: https://www.felesatra.moe/blog/2026/06/19/robots https://www.felesatra.moe/blog/2026/06/19/robots
- Geezus_42 4mo agoThey still ignore it and scan everything even though you have told them not to scan certain paths.