26 ms·
The deal of the internet has always been: send me what you want and I’ll render it however I want. This includes feeding it into AI bots now. I don’t love being
by bee_rider 2y ago
The deal of the internet has always been: send me what you want and I’ll render it however I want. This includes feeding it into AI bots now. I don’t love being on the same side as these “AI” snakeoil salesmen, but they are following the rules of the road.
Robots.txt is just a voluntary thing. We’re going to see more and more of the internet shut off by technical means instead, which is a bummer. But on the bright side it might kill off the ad based model. Silver linings and all that.