3 ms·
Most Top News Sites Block AI Bots. Right-Wing Media Welcomes Them
- bookofjoe 3y agohttps://archive.ph/rGdYL https://archive.ph/rGdYL
- skymast 3y ago[dead]
- h2odragon 3y ago> Originality tallied which sites block GPTbot and other AI scrapers by surveying the robots.txt files that websites use to inform automated web crawlers which pages they are welcome to visit or barred from. Amazing how much you can tell from someone's `robots.txt` file... What does the complete lack of such a file on my site mean? Am I welcoming the AI overlords or fighting the Nazi censors?