18 ms·

Lets not try to qualify the wrongs by picking a metric and evaluating just one side of it. A static website owner could be running with a very small budget and

Lets not try to qualify the wrongs by picking a metric and evaluating just one side of it. A static website owner could be running with a very small budget and the scraping from bots can bring down their business too. The chances of a static website owner burning through their own life savings are probably higher.

expedition32 7mo ago
Perhaps the long play is to destroy all small hobby websites until only a AI directed web is left.
miki123211 7mo ago
If you're truly running a static site, you can run it for free, no matter how much traffic you're getting. Github pages is one way, but there are other platforms offering similar services. Static content just isn't that expensive to host. THe troubles start when you're actually running something dynamic that pretends to be static, like Wordpress or Mediawiki. You can still reduce costs significantly with CDNs / caching, but many don't bother and then complain.
- jazzyjackson 7mo ago
  It's true it can be done but many business owners are not hip to cloudflare r2 buckets or github pages. Many are still paying for a whole dedicated server to run apache (and wordpress!) to serve static files. These sites will go down when hammered by unscrupulous bots.
- ezrast 7mo ago
  Setting aside the notion that a site presenting live-editability as its entire core premise is "pretending to be static", do the actual folks at Wikimedia, who have been running a top 10 website successfully for many years, and who have a caching system that worked well in the environment it was designed for, and who found that that system did not, in fact, trivialize the load of AI scraping, have any standing to complain? Or must they all just be bad at their jobs? https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operations-of-the-wikimedia-projects/ https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-th...
- arminsergiony 7mo ago
  [dead]

expedition32 7mo ago

Perhaps the long play is to destroy all small hobby websites until only a AI directed web is left.

miki123211 7mo ago

If you're truly running a static site, you can run it for free, no matter how much traffic you're getting. Github pages is one way, but there are other platforms offering similar services. Static content just isn't that expensive to host. THe troubles start when you're actually running something dynamic that pretends to be static, like Wordpress or Mediawiki. You can still reduce costs significantly with CDNs / caching, but many don't bother and then complain.

jazzyjackson 7mo ago
It's true it can be done but many business owners are not hip to cloudflare r2 buckets or github pages. Many are still paying for a whole dedicated server to run apache (and wordpress!) to serve static files. These sites will go down when hammered by unscrupulous bots.
ezrast 7mo ago
Setting aside the notion that a site presenting live-editability as its entire core premise is "pretending to be static", do the actual folks at Wikimedia, who have been running a top 10 website successfully for many years, and who have a caching system that worked well in the environment it was designed for, and who found that that system did not, in fact, trivialize the load of AI scraping, have any standing to complain? Or must they all just be bad at their jobs? https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-the-operations-of-the-wikimedia-projects/ https://diff.wikimedia.org/2025/04/01/how-crawlers-impact-th...
arminsergiony 7mo ago
[dead]