3 ms·

Put a CloudFlare web application firewall at the front of the site and then use its rate limited / CAPTCHA features to throttle traffic. It is the easiest way t

Put a CloudFlare web application firewall at the front of the site and then use its rate limited / CAPTCHA features to throttle traffic. It is the easiest way to get rid of parasitic scraping and API abuse. Cost is $0.

marginalia_nu 4y ago
Yeah, that's essentially what I've done, except I'm paying for their cheapest non-free tier to have a bit more control over it. I really wish I didn't have to route all my traffic through an untrusted a 3rd party like that, but I guess we can't have nice things on the Internet anymore.
- B1FF_PSUVM 4y ago
  > I guess we can't have nice things on the Internet anymore Not since it left the larval stage and became "pay for play", no. Oh, well, those taxpayer-funded years were nice for those of us who were around.
  - bcrosby95 4y ago
    I think I remember wondering, after the dotcom bust, if the whole web thing would actually take off. The reasoning I vaguely remember reading was that the internet required government subsidy to exist - at first directly, then in the form of universities, and the bust was a sign that it couldn't exist without one. I don't remember how prevalent the view was at the time though. Obviously it turned out to be wrong.
- kordlessagain 4y ago
  Putting authentication on the site would be easier. There a rub here, in that people expect to search things without being logged in. But then if you don't log in people, anyone can come calling, including bots. This then causes you to do things like get a third party to filter the data, which then affects the users by having to reroute their traffic to someone else to get rid of some of the visits you don't want from the bots. And round and round. Simple authentication to the site with tokens might solve the problem. If an IP comes calling that does so with out authentication, or payment, then hang the connection.
Schroedingersat 4y ago
> Cost is $0. Cost is the slow enclosure of the internet hy a handful of giant companies and once attestation is universal having anyone without a locked down device be locked out of most of the internet without providing endless free labour. FTFY

marginalia_nu 4y ago

Yeah, that's essentially what I've done, except I'm paying for their cheapest non-free tier to have a bit more control over it. I really wish I didn't have to route all my traffic through an untrusted a 3rd party like that, but I guess we can't have nice things on the Internet anymore.

B1FF_PSUVM 4y ago
> I guess we can't have nice things on the Internet anymore Not since it left the larval stage and became "pay for play", no. Oh, well, those taxpayer-funded years were nice for those of us who were around.
- bcrosby95 4y ago
  I think I remember wondering, after the dotcom bust, if the whole web thing would actually take off. The reasoning I vaguely remember reading was that the internet required government subsidy to exist - at first directly, then in the form of universities, and the bust was a sign that it couldn't exist without one. I don't remember how prevalent the view was at the time though. Obviously it turned out to be wrong.
kordlessagain 4y ago
Putting authentication on the site would be easier. There a rub here, in that people expect to search things without being logged in. But then if you don't log in people, anyone can come calling, including bots. This then causes you to do things like get a third party to filter the data, which then affects the users by having to reroute their traffic to someone else to get rid of some of the visits you don't want from the bots. And round and round. Simple authentication to the site with tokens might solve the problem. If an IP comes calling that does so with out authentication, or payment, then hang the connection.

Schroedingersat 4y ago

> Cost is $0. Cost is the slow enclosure of the internet hy a handful of giant companies and once attestation is universal having anyone without a locked down device be locked out of most of the internet without providing endless free labour. FTFY