2 ms·
Cloudflare runs DDoS protection against a surprisingly many sites now. Often you don't even see the "checking your browser" screen, it just happens passively.
by echelon 5y ago
Cloudflare runs DDoS protection against a surprisingly many sites now. Often you don't even see the "checking your browser" screen, it just happens passively.
While individual authors and publishers benefit substantially (caching, anti-DDoS, etc.), this is a huge net loss for the internet. It essentially entrenches Google and the rest of MAGMA as the forever winners of spidering the web. If you try to spin up your own crawler, you'll get hit by this bullshit. It's becoming impossible to bootstrap now.