3 ms·
I hate to be a pedant but I see it used wrong a lot. "DDOS" stands for Distributed denial of service, specifically indicating the traffic is coming from many so
by RugnirViking 3y ago
I hate to be a pedant but I see it used wrong a lot. "DDOS" stands for Distributed denial of service, specifically indicating the traffic is coming from many sources (wide range of ip), which is what makes it so hard to defend against. Someone scraping too fast would be performing an unintentional DOS, because they probably arent scraping using a botnet (and if they were they probably do in fact intend to attack)
- jakear 3y agoAnd if they’re scraping using “serverless” compute? It’s distributed, but could be used without “meaning” to attack. That seems to be what happened to Archive (they said AWS, could be ECS or Lambda, idk). FWIW this thread inspired me to implement Cloudflare Turnstile on one of my pages - highly recommend. As compared to reCAPTCHA your users are never wasting their life away clicking on traffic signs, and as compared to mCaptcha you don’t have to host the server yourself.
- geraldwhen 3y agoExtremely unlikely that was lambda unless the user liked to light money on fire.
- jakear 3y agoWell they were an "AI Startup"....
- marginalia_nu 3y agoIn practice they're often using botnets. It's what makes this such a difficult problem to deal with. If they weren't it would be trivial to rate-limit or block problematic IPs or subnets.
- RugnirViking 3y agoas I mentioned in the post: why would a legitimate scraper be using a botnet?
- marginalia_nu 3y agoIn most cases, they aren't legitimate scrapers. They have a complete disregard for the resources they use from the websites they scrape and use a botnet to hide their point of origin.
- tiagod 3y agoEven if their scraping operation is centralized, it's common to use residential proxy providers which spread the activity over a big number of IPs.
- tjr225 3y ago[dead]