I wanted to serve content to a regional area but avoid discovery by spammers and wasting bandwidth on crawlers. I didn’t need SEO as I had a separate channel t

I wanted to serve content to a regional area but avoid discovery by spammers and wasting bandwidth on crawlers. I didn’t need SEO as I had a separate channel to notify users my site existed. At first I thought I’d blacklist the bots: no Google, no bing, no OpenAI. But I realised that a whitelist would be much more effective. The process I used was to gather the ASNs of every local ISP where my users are. Thanks to BGP looking glass services, you can convert this to a list of every advertised prefix. You can coalesce this into an optimised data structure using “ipset” to build a firewall rule that affects only addresses you whitelist. I also found a data center offering colocation which is directly peered with every major ISP in my area and a few of the minor ones. Any visitor that doesn’t match my whitelist sees a decoy page. For the protected content, I set IP TTL just high enough to deliver only within the service area. One drawback of this approach is it’s difficult to include local mobile users (without including their entire parent network) because they will need higher TTLs and come from a network with many prefixes. I’d be interested in an iptables rule that can drop connections after a few packets based on an RTT heuristic. This approach is the opposite of “zero-trust” and it fails as soon as your threats start moving into into your user’s network.