inigyou

5 ms·

91.
by inigyou 2mo ago
No CDN blocks bots. Some claim to. None do.
92.
by inigyou 2mo ago
What if it's really obvious they didn't read the article?
93.
by inigyou 2mo ago
In fact they already do this. Buying 100 android phones and chargers is cheaper than reverse engineering whatever you're trying to automate - or was, before AI.
94.
by inigyou 2mo ago
Cloudflare doesn't block bots.
95.
by inigyou 2mo ago
Everyone ignores robots.txt because every robots.txt blocks everyone.
96.
by inigyou 2mo ago
Especially since if any crawler IP is in your country, you can get a subpoena to find out who it is.
97.
by inigyou 2mo ago
Bright Data's business is legal, but they could be subpoenaed to find out which customer is making these requests, but first you would have to prove they were actually involved, because there are many residential proxy providers.
98.
by inigyou 2mo ago
How do you know who the bad actors are, if you don't use the legal process to unmask them?
99.
by inigyou 2mo ago
Sometimes the first step of a lawsuit is discovering who you're suing. It's not unusual and there are processes for it. You could bring something like an access log to a court and receive an order for all ISPs involved to unmask t
100.
by inigyou 2mo ago
No point, they are all unique requests.
101.
by inigyou 2mo ago
Put a cookie wall in front. The bot will either load the cookie and have a persistent identifier, or not load the cookie and not get in
102.
by inigyou 2mo ago
No they mean the people doing the actual work, I think.
103.
by inigyou 2mo ago
I'm one of the 5% apparently, cloudflare thinks I'm a bot. I suppose you'll exempt me in exchange for all my ID documents and bank statements?
104.
by inigyou 2mo ago
Monero would work better.
105.
by inigyou 2mo ago
You try hosting gitea in 2026. The only other option is taking the site offline.
106.
by inigyou 2mo ago
You just take a sample of the crawling to whichever police department knows how to deal with DDoS, they subpoena the ISPs involved and trace who's using the proxies?
107.
by inigyou 2mo ago
The internet is already like that, but for some reason the payment only extends as far as the recipient's ISP, not the actual recipient. Most senders pay a flat rate, but their ISP doesn't.
108.
by inigyou 2mo ago
abuseipdb.com Some ISPs ban customers based on a single report there - have fun!
109.
by inigyou 2mo ago
It's running a diff between two arbitrary blobs of text. Do you actually have a solution or are you just saying to remove the feature from the site entirely? > Us PHP developers I can tell.
110.
by inigyou 2mo ago
Or poll a merkle tree root hash, like, um, git
111.
by inigyou 2mo ago
That would be a different bot, one written specifically for your site. Mainly we're discussing the dumb ones that just crawl all possible http links
112.
by inigyou 2mo ago
Once detected, don't block them because they'll just change strategies automatically, but you can toy with them, like returning a page full of random numbers instead of real data.
113.
by inigyou 2mo ago
It isn't illegal, there is no law against "network intrusion" which is a term you just made up, and if it was a real term it probably wouldn't cover this. There are laws against things like "unauthorized access to a
114.
by inigyou 2mo ago
3) just track down the people and arrest them. DDoS is already a felony, and residential proxies aren't untraceable to law enforcement.
115.
by inigyou 2mo ago
What's actually stopping them isn't the PoW, it's the customisation effort. If one site has Anubis nothing scrapes it. If many sites have Anubis they write counter scrapers. Today if you make a slight change to the Anubis alg
116.
by inigyou 2mo ago
And the residential proxy business was created by Cloudflare, who was created by us using Cloudflare. I've been on all three sides (user of RPs, getting paid to run an RP, and trying to block RPs from my site). Residential proxy servic
117.
by inigyou 2mo ago
Yes they are. The proxy is just a proxy. All processing is central.
118.
by inigyou 2mo ago
Both groups exist. Scraper DDoS is likely to burn 100% of your CPU on git diffs if you host git. But static file sites are unlikely to notice it.
119.
by inigyou 2mo ago
Yes. Any number of methods work. From requiring a cookie to requiring CSS to be loaded to a PoW to a JavaScript redirect. Most scraper bots are really dumb. If you do it cookie based, you can also put a poison link on every page that invali
120.
by inigyou 2mo ago
Alternative step 3: arrest the asshole. We don't even need a new law. What they're doing is already a felony, and there's a process already in place to go to a court and request unmasking of 10000 IP addresses.

More ›