3 ms·

And whilst the IA will honour requests not to archive/index, more aggressive scrapers won't, and will disguise their traffic as normal human browser traffic. S

And whilst the IA will honour requests not to archive/index, more aggressive scrapers won't, and will disguise their traffic as normal human browser traffic. So we're basically decided we only want bad actors to be able to scrape, archive, and index.

JumpCrisscross 9mo ago
> we're basically decided we only want bad actors to be able to scrape, archive, and index AI training will be hard to police. But a lot of these sites inject ads in exchange for paywall circumvention. Just scanning Reddit for the newest archive.is or whatever should cut off most of the traffic.
mananaysiempre 9mo ago
> If you find yourself wondering, or just feeling, "Why is everyone I wind up dealing with an asshole?" you might want to consider the possibility that you have set up an asshole filter. https://siderea.dreamwidth.org/1209794.html https://siderea.dreamwidth.org/1209794.html
- trollbridge 9mo ago
  It’s hard to filter out legit looking traffic tho.