3 ms·
Curious to see if ignoring robots.txt, which has almost miraculously held up for decades, could eventually lead to a lawsuit.
by winstonp 2y ago
Curious to see if ignoring robots.txt, which has almost miraculously held up for decades, could eventually lead to a lawsuit.