2 ms·
Totally agree with the sentiment and the project linked at the bottom of the article is pretty neat. It does feel a bit hopeless to rely on robots.txt to deter
by nelsonfigueroa 3y ago
Totally agree with the sentiment and the project linked at the bottom of the article is pretty neat. It does feel a bit hopeless to rely on robots.txt to deter AI bots but I guess it's better than nothing. Appreciate the effort.